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treatment planning, appliance design and appliance 
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acquires images of the dentition which arc converted to 
three-dimensional frames of data. The data from the 
several frames are registered to each other to provide a 
complete three-dimensional virtual model of the dentition. 
Individual tooth objects are obtained from the virtual 
model. A computer-interactive software program provides 
for treatment planning, diagnosis and appliance from the 
virtual tooth models. A desired occlusion for the panent is 
obtained from the treatment planning software. The virtual 
model of the desired occlusion and the virtual model of the 
original dentition provide a base of information for custom 
manufacture of an orthodontic appliance. A variety of 
possible appliance and appliance manufacturing systems 
are contemplated, including customized archwires and 
customized devices for placement of off-the shelf brackets 
on the archwires, and removable orthodontic appliances. 
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BACKGROUND OF THE INVEINTION 

A. Field of the Invention 

5 This invention relates generally to the field of orthodontics. More particularly, the 

invention relates to a computerized, interactive method and associated system for 
orthodontic treatment. The system includes a hand-held optical scanner capturing 3- 
dimensional information of objects, interactive computer-based treatment planning using 
three-dimensional tooth objects and user specified simulation of tooth movement, and 
10 apphance manufacturing apparatus, including bending machines. 

B. Description of Related Art 

In orthodontics, a patient suffering firom a malocclusion is typically treated by 
bonding brackets to the surface of the patient's teeth. The brackets have slots for receiving 
15 an archwire. The bracket-archwire interaction governs forces applied to the teeth and 
defines the desired direction of tooth movement. Typically, the bends in the wire are made 
manually by the orthodontist. During the course of treatment, the movement of the teeth is 
monitored. Corrections to the bracket position and/or wire shape are made manually by the 
orthodontist. 
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The key to efBciency in treatment and maximum quality in results is a reaKstic 
simulation of the treatment process. Today's orthodontists have the possibility of taking- 
plaster models of the upper and lower jaw, cutting the model into single tooth models and 
sticking these tooth models into a wax bed, lining them up in the desired position, the so- 
5 called set-up. This approach allows for reaching a perfect occlusion without any guessing. 
The next step is to bond a bracket at every tooth model. This would tell the orthodontist the 
geometry of the wire to run through the bracket slots to receive exactly this result. The next 
step involves the transfer of the bracket position to the original malocclusion model. To 
make sure that the brackets will be bonded at exactly this position at the real patient's teeth, 
10 small templates for every tooth would have to be fabricated that fit over the bracket and a 
relevant part of the tooth and allow for reliable placement of the bracket on the patient's 
teeth. To increase efficiency of the bonding process, another option would be to place each 
single bracket onto a model of the malocclusion and then fabricate one single transfer tray 
per jaw that covers all brackets and relevant portions of ev^y tooth. Using such a transfer 
15 tray guarantees a very quick and yet precise bonding using indirect bonding. 

However, it is obvious that such an approach requires an extreme amoxmt of time 
and labor and thus is too costly, and this is the reason why it is not practiced widely. The . 
normal orthodontist does not fabricate set-ups; he places the brackets directly on the 
patient's teeth to the best of his knowledge, uses an off-the-shelf wire and hopes for the 
20 best. There is no way to confirm whether the brackets are placed correctly; and 
misplacement of the bracket will change the direction and/or magnitude of the forces 
imparted on the teeth. "While at the beginning of treatment things generally run well as all 
teeth start to move at least into the right direction, at the end of treatment a lot of time is lost 
by adaptations and corrections required due to the fact that the end result has not been 
25 properly planned at any point of time. For the orthodontist this is still preferable over the 
lab process described above, as the efforts for the lab process would still exceed the efforts 
that he has to put in during tireatinent. And the patient has no choice and does not know tiiat 
treatment time could be significantly reduced if proper planning was done. 
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U.S. Patent 5,431,562 to Andreiko et al. describes a computerized, appliance-driven 
approach to orthodontics. In this method, &st certain shape information of teeth is 
acquired. A uniplanar target archform is calculated from the shape information. The shape 
of customized bracket slots, the bracket base, and the shape of an orthodontic archwire, are 
5 calculated in accordance with a mathematicaUy-derived target archform. The goal of the 
xAndreiko et al. method is to give more predictability, standardization, and certainty to 
orthodontics by replacing the human element in orthodontic appliance design with a 
deterministic, mathematical computation of a target archform and appliance design. Hence 
the '562 patent teaches away from an interactive, computer-based system in which the 

10 orthodontist remains fully involved in patient diagnosis, appliance design, and treatment 
plamring and monitorihg. 

More recently, in the late 1990's Align Technologies began offering transparent, 
removable ahgning devices as a new treatment modality in orthodontics. In this system, a 
plaster model of the dentition of the patent is obtained by the orthodontist and shipped to a 

15 remote appliance manufacturing center, where it is scanned with a laser. A computer 
model of the dentition in a target situation is generated at the appliance manufacturing 
center and made available for viewing to the orthodontist over the Internet The 
orthodontist indicates changes they wish to make to individual tooth positions. Later, 
another virtual model is provided over the Internet and the orttiodontist reviews the revised 

20 model, and indicates any further changes. After several such iterations, the target situation 
is agreed upon. A series of removable aligning devices or shells are manufactured and 
delivered to the orthodontist. The shells, in theory, will move the patient's teeth to the 
desired or target position. 
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The art has lacked an effective, computer-based interactive orthodontic treatment 
planning system that provides the necessary tools to allow the orthodontist to quickly and 
efficiently design a treatment plan for a patient. The art has also lacked a treatment 
planning system in which the orthodontist-derived parameters for the treatment can be 
5 translated into a design of an orthodontic appliance in real time, while the patient is in the 
chair. Real-time appUance design as described herein also allows for real-time 
communication of the treatment plan or appliance design to occur with the patient, or 
transmitted over a conmiunications link and shared with a colleague or remote appUance 
manufacturing facihty. Alternatively, the treatment planning can be performed remotely 
10 and a digital treatment plan sent to the orthodontist for review, interactive modification, or 
approval. 

Scanners are devices for capturing and recording information firom a surface of an 
object Scanners for obtaining information from a two-dimensional surface, such as reading 
bar codes or characters printed on a piece of paper, are widely known. Several scanners 
15 have been proposed for recording three-cKmensional information as well, including the field 
of dentistry. 

U.S. Patent 4,837,732 and U.S. Patent 4,575,805 to Brandestini and Moermann 
propose a scanning system for in vivo, non-contact scanning of teeth. The patents describe a 
procedure for optically mapping a prepared tooth with a non-contact scan-head. The scan- 

20 head deUvers the contour data, converted to electrical format, to be stored in a memory. A 
computer reads the memory following a line scan pattern. A milling device is slaved to 
follow this pattern by means of position control signals and mills an implant for the 
prepared tooth cavity. 

The scan-head of the '732 and *805 patents includes a light emitting diode, with 

25 integral lens that radiates light onto the cavity. Before reaching the object, the rays of light 
are reflected by a mirror and pass through a ruling consisting of a plxirality of parallel slits, 
or an alternating pattem of parallel opaque and transparent stripes. The reflected tight is 
focused by a lens onto a charge-coupled device (CCD) sensor. Depth information is 
determined in accordance with a principle known as "active triangulation," using parameters 

30 shown in Figure 9 of this document and described subsequently. Basically, the object is 
viewed under an angle different from the incident rays due to a parallax effect. Each light 
stripe will have an apparent positional shift and the amount of the shift at each point along 
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each light stripe is proportioiial to the vertical height of the corresponding portion of the 
surface on the object. 

U,S. Patent No. 5.372,502 to Massen et al. describes an optical probe for measuring 
teeth that works on a similar principle. As noted in the Massen et al. patent, the Brandestini 
5 et al. technique is difficult to use when there are large variations in surface topogr^hy 
since such large jumps displace the pattern by an amount larger than the phase constant of 
tiae pattern, rnaldng it difficult to reconstruct the pattern of lines. Furthermore, precise 
knowledge of the angle of incidence and angle of reflection^ and the separation distance 
between the light source and the detector, are needed to make accurate determinations of 
10 depth. Furthermore, the scanner has to be rather carefully positioned with respect to the 
tooth^and would be unable to make a complete model of the dentition. 

U.S. Patent Nos. 5,027,281 to Rekow et al. describes a scanning method using a 
three axis . positioning head with a laser source and detector, a rotational stage and a 
computer controller. The computer controller positions both the rotational stage and the 
15 positionmg head. An object is placed on the rotational stage and the laser beam reflects 
from it. The reflected laser beam is used to measure the distance between the object and the 
laser source. X and Y coordinates are obtained by movement of the rotational stage or the 
positioning head. A three-dimensional virtual model of the object is created from the laser 
scanning. The '281 patent describes using this scanning method for scanning a plaster 
20 model of teeth for purposes of acquiring shape of the teeth to form a dental prosthesis. The 
system of the '281 patent is not particularly flexible, since it requires the object to be placed 
on the rotational stage and precise control of the relative position of the object and the 
positioning head is required at all times. It is imsuited for in \nvo scanning of the teeth. 

U.S. Patent 5,431,562 to Andreiko et al. describes a method of acquiring certain 
25 shape information of teeth from a plaster model of the teeth. The plaster model is placed 
on a table and a picture is taken of the teeth using a video camera positioned a known 
distance away from the model, looking directly down on the model. The image is 
displayed on an input computer and a positioning grid is placed over the image of the teeth. 
The operator manually inputs X and Y coordinate information of selected points on the 
30 teeth, such as the mesial and distal contact points of the teeth. An alternative embodiment is 
described in which a laser directs a laser beam onto a model of the teeth and the reflected 
beam is detected by a sensor. The patent asserts, that three-dimensional information as to 
teeth can be acquired from this technique but does not explain how it would be done. 

- 6 • 
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Neither of the techniques of Andreiko have met with widespread commercial success or 
acceptance in orthodontics. Neither technique achieves in vivo scanning of teeth. 
Moreover, the video technique does not produce complete three-dimensional information as 
to the teeth, but rather a limited amount of two-dhnensional information, requiring 
significant manual operator input. Even using this technique, additional equipment is 
required even to describe the labial surface of a tooth along a single plane. 

The art has also lacked a reliable, accurate, low-cost, and easHy used scanning 
system that can quickly and automaticaUy acquire three-dimensional information of an 
object, without requiring substantial operator input, and in particular one that can be held in 
the hand and used for in vivo scanning or scanning a model. The present invention meets 
this need. 
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SmVEVIARY OF THE INVENTION 

An interactive, orthodontic care system is provided based on scanning of teeth. The 
system includes a hand-held scanner and associated processing system for capturing images 
of the dentition of the patient and processing the images to generate a full, virtual, three- 
dimensional model of the dentition. A data conditioning system processes the virtual, three- 
dimensional model and responsively generates a set of individual, virtual three-dimensional 
tooth objects representing teeth in the dentition of the patient A workstation is provided 
having a user interface for display of the set of individual, virtual three-dimensional tooth 
objects. Interactive treatment planning software is provided on the workstation permitting a 
user to manipulate the virtual three-dimensional tooth objects to thereby design a target 
situation for the patient in three dimensions and parameters for a customized orthodontic 
appUance for the teeth. 

The hand-held scanner and associated processing system, data conditioning system, 
and workstation may all he installed in an orthodontic clinic- Alternatively, the hand-held 
scanner and workstation are iustalled in an orthodontic clinic, and wherem the data 
conditioning system is mstaUed m a general purpose computer at a remote location from 
orthodontic clinic. Further, the treatment planning software can be either installed at the 
clinic, and/or at a remote location, and/or at a precision appliance manufacturing center that . 
manufactures a custom orthodontic apphance. The type of appliance may vary 
considerably. 

The treatment planning apparatus can be considered an interactive, computer-based 
computer aided design and computer aided manufacturing (CAD/CAM) system for 
orthodontics. The apparatus is highly interactive, in that it provides the orthodontist with 
the opportunity to both observe and analyze the current stage of the patient^s condition and 
develop and specify a target or desired stage. A shortest direct path of tooth movement to 
the target stage can also be deteraiined. Further, the apparatus provides for simulation of 
tooth movement between current and target stages. 

In its broader aspects, the apparatus comprises a workstation having a processmg 
imit and a display, and a memory storing a virtual, complete tiiree-dimensional model 
representing the dentition of a patient. The vumal three-dimensional model can be obtained 
from one of several possible sources; in the preferred embodiment it is airived at from a 
scanning of the dentition. The apparatus fiirther includes software executable by the 
processuig unit that accesses the model and displays the model on the display of the 
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workstation. The software further includes navigation tools, e.g., typed commands, icons 
and/or graphical devices superimposed on the displayed model, that enables a user to 
manipulate the model on the display and simulate the movement of at least one tooth in the 
model relative to other teeth in the model in three-dimensional space, and quantify the 
5 amount of movement precisely. This simulation can be used, for example, to design a 
particular target situation for the patient. 

The development of a unique target situation for the patient has utility in a variety of 
different orthodontic appliances, including an approach based on off-the-shelf or generic 
brackets and a custom orthodontic archwire. The scope of the invention is sufficient to 

10 encompass other types of appliances, such as an approach based on customized brackets, 
retainers, or removable aligning devices. In a bracket embodiment, the memory contains a 
library of virtual, three-dimensional orthodontic brackets. The software permits a us^ to 
access the virtual brackets through a suitable screen display, and place the virtual brackets 
on the virtual model of the dentition of the patient. This bracket bonding position can be 

15 customized on a tooth by tooth basis to suit individual patient anatomy. Because the tooth 
models, brackets and archwire are individual objects, and stored as such in memory, the 
treatment planning apparatus can simultaneously display the virtual brackets, the archwire 
and the virtual model of the dentition, or some lesser combination, such as just the brackets, 

just the dentition, or the brackets and the archwire but not the teeth. The same holds 

20 tme with other appliance systems. 

In a preferred embodiment, the virtual model of teeth comprises a set of virtual, 
individual three-dimensional tooth objects. A method of obtaining the tooth objects from a 
scan of teeth, and obtaining other virtual objects of associated anatomical stractures, e.g., 
gums, roots and bone is described. When the teeth are separated from each other and from 

25 the gums, they can be individually manipulated. Thus, the individual tooth objects can be 
uidividually selected and moved relative to other teeth in the set of virtual tooth objects. 
This feature permits individual, customized tooth positioning on a tootii by tooth basis. 
These positioning can be in terms or angular rotation about three axis, or translation in 
transverse, sagittal or coronal planes. Additionally, various measurement features are 

30 provided for quantifying the amount of movement. 

One of the primary tools in the treatment planning apparatus is the selection and 
customization or a desired or target archfomi. Again, because the teeth are individual tooth 
objects, they can be moved independently of each other to define an ideal arch. This 
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development of the target archform could be calculated using interpolation or cubic spline 
algorithms. Alternatively, it can be customized by the user specifying a type of archform 
(e.g. Roth), and the tooth are moved onto that archform or some modification of that 
archform. Ilie archform can be shaped to meet the anatomical constraints of the patient 
5 After the initial archform is designed, the user can again position the teeth on the archform 
as they deem appropriate on a tooth by tooth basis. The treatment planning software thus 
enables the movement of the virtual tooth objects onto an archform which may represent, at 
least in part, a proposed treatment objective for the patient. 

Numerous other features are possible with the treatment planning software, 

10 including movement of the teeth with respect to the other teeth in the archform, changing 
the position of the virtual brackets and the teeth with respect to each other, or opposing teeth 
with respect ot the selected archform. Custom archwire bends can be simulated to provide 
additional corrections. Bonding corrections at the bracket-tooth interface are also possible. 
In another aspect of tiie invention, a method is provided for digital treatment 

15 planning for an orthodontic patient on a workstation having a processiug unit, a user 
interface including a display and software executable by the processing unit. The method 
comprises the steps of obtaining and storing a three-dimensional virtual model of teeth 
representing the dentition of the patient in a current or observed situation. The virtual model 
is displayed on the display. The method further includes the step of moving the position of 

20 teeth in the virtual model relative to each other so as to place the teefli of the virtual model 
into a target situation and displaying the virtual model with the teeth moved to the target 
situation to the user. Parameters for an orthodontic appliance to move the patient's teeth 
firom the current situation to the target situation can be derived Scorn the virtual model and 
the target situation. For example, if virtual brackets are placed on the teeth, their location 

25 in the target situation can dictate the design of an archwire to move the teeth to the target 
situation. 

. The scanner system is provided for capturing three-dimensional information of an 
object. The object can be virtually any object under scrutiay, however the present document 
will describe an application in which the object is the dentition of a patient suffering firom a 
30 malocclusion. 

The scanning system enables threeniimensional surface information to be obtained 
with a very high decree of precision. Moreover, the scanning system can be used without 
requiring precise movement of the scanner, or requiring the object under scratiny to be fixed 
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in space. Surprisingly, the scanner is able to generate precise three dimensional surface 
information by simply moving the scanner over the surface of the object, such as by hand, in 
any manner that is convenient for the user, even if the object moves in any random direction 
during the scanning within reasonable limits. Thus, the scanner can be used to capture the 
5 surface of a patient's dentition in a minute or two, even if the patient moves their head or 
jaw while the scanning is occurring. Precise knowledge of the spatial relationship between 
the seamier and the object is not required. 

The scanner obtains a set of images, which are processed in a computer to calculate 
the surface configuration of the object in three dimensions of space automatically, quickly, 
10 with high precision, and with essentially no human involvement other than the act of 
scanning. The precision or accuracy will be dictated largely by the extent to which the 
object under scrutiny tends to have undercut or shadowed features which are difficult to 
detect, necessitating a narrow angle between the projection and imaging axes. For teeth, an 
accuracy of under 20 or 30 microns is possible. This accuracy can be further improved 
15 depending on the nature of the surface, such as if the surface does not have a lot of undercut 
or shadowed features, by increasing the angular separation of the projection axis and the 
imaging axis. 

Each image captured by the scanner is converted to a virtual, three-dimensional 
point cloud or "fi-ame." The illustrated embodiment has a relatively coarse resolution for 
20 any single frame, due to a coarse projection pattern, but a fine resolution is obtained by 
obtaining multiple images and performing a registration procedure on the fi^es, as 
described below. Since each point on the surface of the object is captured in a pliffaUty of 
images (such as five or six in a typical example of scanning), the registration of fi^es 
results in a fine resolution. An even finer resolution can be obtained by scanning slower and 
25 capturing more images of the surfece of the object from different perspectives and 
registering the resulting firames to each other. 

This surface configuration of the object in three dimensions of space can be 
represented as a mathematical model, i.e., a virtual model of the object, which can be 
displayed on any workstation or computer using available software tools. The 
mathematical model can be viewed in any orientation in space, permitting detailed analysis 
of the surface. The model can be compared to template objects stored in a computer. 
Deviations in the object from the template can be quantified and analyzed. Further, the 
virtual model can be transported from one computer and another computer anywhere in the 
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world essentially instantaneously over communications links such as the Ihtemet The 
model can be rephcated in a computer and thus shared and used by multiple users 
simultaneously. 

The scaimer system is useful for a wide variety of industrial, medical, archeological, 
forensic, archival, or other purposes. Furthermore, the scanner can be scaled down in size 
such that it can be hand-held and used to scan small objects, e.g., teeth or small machined 
parts, or scaled up in size so that it can be used to make mathematical models of larger scale 
objects such as works of art, sculptures, archeological sites (e.g., the caves at Lascaux, 
France or the dwellings or kivas in Mesa Verde National Park), rooms or building facades. 

In accordance with a preferred embodiment, the scanner system includes a projection 
system that projects a pattern onto the object along a first optical axis. The pattern may 
consist of parallel lines, parallel lines separated by shapes or colors, such as colored dots, or 
other suitable pattern. The projected pattern is used to gather information as to the surface 
characteristics of the object in accordance with the metiiods and procedures described in 
more detail below! - 

The scanner further includes an electronic imaging device, preferably in the form of 
a charger coupled- device comprising a two-dimensional array of photo-sensitive pixels. The 
electronic imaging device is oriented along a second optical axis different from the first 
optical axis. The electronic imaging device forms an image of the pattern after reflection of 
the pattern off of the object vmder scrutiny. The surface configuration of the object will 
cause the projection pattern to become distorted and changed, and thereby provide 
infomaation as to the surface configuration. This information as to the surfece is captured 
by the imaging device as two-dimensional images of the reflection pattem. These images 
are processed in accordance with procedures described herein to derive three-dimensional 
information as to the surface of the object. 

The scanning system, in a preferred embodiment, further includes a memory that 
stores data representing a three axis (X, Y, Z) cahbration relationship for the scanner. The 
calibration relationship can be in the form of a table or in the form of a mathematical 
function (e.g., polynomial, spline, or otha: function). The cahbration relationship identifies 
two properties of the scaimer. (1) pixel coordinates for the electronic imaging device for 
nimierous portions of the pattern, said pixel coordinates associated with distance 
information from the projection system in a Z direction at at least two different Z distances, 
and (2) distance information in X and Y directions, for the mmierous portions of said 
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pattern, at the at least two different Z distances. A mettiod of obtaining the calibration 
relationship is described in detail below. While the simplest form of the relationship is a 
table, as described in detail below, these calibration relationships could be equivalently 
represented by one or more mathematical functions as will be apparent to those skilled in 
art. 

The calibration relationship is used to derive three-dimensional coordinates for 
points on the surface of an object imaged by the imaging device. The generation and use 
of the preferred calibration table is also explained in further detail below. Other cahbration 
tables or procedures are also possible. The use of the calibration relationship allows the 
scanner to operate without precise knowledge of the optical and mechanical properties of 
the scaimer, as is required in prior art systems. 

The scaiming system further includes at least one data processing unit, e.g., the 
central processing unit of a computer or a digital signal processor, which processes the 
images of the projection pattern after reflection from the surface of the object. The 
processing unit compares data from the image (e.g., pixel locations where certain points m 
the projection pattern are imaged) to the calibration relationship to thereby derive spatial 
information, in three dimensions, of points on the object reflecting the projected pattern 
onto the elecfronic imaging device. Multiple processing units can be used to reduce the 
amount of time it takes to process the two-dimaisional images, calculate three-dimensional 
coordinates for points in each image, aad register frames of three-dimensional coordinates 
relative to each other to generate a complete virtual model of the object. 

The scanning system thus derives three-dimensional information of the object from 
the images generated by the imaging device and from the calibration relationship stored in 
memory. Precise knowledge of the optical characteristics of the scanner, the angles between 
the first and second optical axes, the angle between the optical axis and the point in the 
imaging device, or the separation distance of the projection device from the imaging device 
are not necessary. The projection system and the imaging device can be even uncoupled 
relative to each other. The calibration relationship automatically and completely 
compensates for these issues, as well as manufacturing variations and tolerances in the 
scanner optics, m a highly reliable manner. Moreover, knowledge of the distance from the 
scanner to the object is not required. Additionally, control over the distance between the 
scanner and the object is not required, within reasonable limits dictated by the depth of 
focus of the imaging optics. Ideally, during scanning the distance from the scaimer to the 
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object is maintained within the limits of the Z distances used during calibration, and that 
distances is within the combined depth of focus of the projection optics and the imaging 
optics. 

The scanning system can be constructed such that the memory, processing unit, and 
opi^voi elements are in a single unit. Alternatively, the processing unit aad memory can be 
located at a separate location, such as a scanning workstation or "scanning node", in order to 
reduce the size of the scanner per se. In such an embodiment, the projection system and the 
image-recording device can be miniaturized into a hand-held scanner device. A suitable 
cable connects the scanner device to the workstation to thereby supply the processing umt 
with scan data, and to receive commands (illumination commands, start/stop commands, 
etc.) from the workstation. 

Preferred appliance manufacturing systems are also described, including bending 
apparatus that bends medical devices, such as orthodontic appliances like archwires, fixation 
plates and other devices. Bracket placement trays are also described. As one embodiment, 
an orthodontic appliance manufacturing system is provided comprising a machine readable 
memory storing, digital data representing a three-dimensional virtual model of a 
malocclusion of a patient and digital information representing the location of orthodontic 
brackets to.be plaqed on the malocclusion, said memdry further storing digital data 
representing a three-dimensional virtual model of the patient's teeth and the dentition and 
location of orthodontic brackets at a target situation in three dimensions. The apparatus 
further includes a. wire - bending robot. A rapid prototyping instrument such as SLA 
(stereolithography) is provided for generating a fliree-dimensional physical model of the 
malocclusion with the brackets placed on the malocclusion. A forming device forms a 
bracket placement tray matching the geometry of the physical model of the malocclusion 
with the brackets. The bracket placement tray may be removed from the physical model, 
said tray formed with spaces for placement of brackets and enabling bonding of the brackets 
to the teeth at a desired location based on the virtual model of the dentition and the 
placement of orthodontic brackets on the malocclusion. 

Numerous other features of the appliance manufacturing apparatus, the treatment 
planning software, the scanrdng system and related features will be more apparent from the 
following detailed description. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is an illustration of an orthodontic care system incorporating a hand-held 
scanner system in accordance with a representative embodiment of the invention. The 
hand-held scanner is used by the orttiodontist or the assistant to acquire fliree-dimensional 
5 information of the dentition and associated anatomical structures of a patient and provide a 
base of information to diagnose and plan treatment for the patient. 

Figure 2 is a block-diagram of a scanning system, suitable for use in the orthodontic 
care system of Figure 1. - - .... 

Figure 3 is a perspective view of a hand-held scanner used to acquire information of 
10 an object under scrutiny, suitable for use in the orthodontic care system of Figure 1. 

Figure 4 is an illusti-ation of a patient being scanned with the hand-held scanner of 
Figure 3. 

Figure 5 is a block diagram of the back office server of Figure 1 showing the 
elemente used to calculate tiie digital model of the patient's dentition and display (he digital 
15 model on a screen display of the server. 

Figure 6 is a flow diagram illustiratmg the sequence of steps used by the processing 
unit in the scaniung station to calculate three-dimensional information of an object from 
images captured by the scanner. 

Figure 7 is a cross-sectional view of the scanner of Figure 3, showing the optical 
20 elements of the projection and imaging aspects of the scanner. 

Figure S is a perspective view of a scanner calibration apparatus of station that is 
used to calibrate the scanner and obtain data for entry in a caUbration table stored in the 
memory of the scanner system. 

Figure 8 A is a perspective view of an alternative embodiment of a calibration device 
25 for the scanner m which the Z direction and X-Y calibration surfaces of Figure 8 are 
combined into a single surface. 

Figure 9 is an illustration of tiie relevant parameters that can be used to calculate 
surface configuration of the object in accordance with a known fashion. 

Figure 9A is an illustration of an electronic imaging device of a scanner, associated 
30 imaging lens system, and an object reflecting a projection pattern onto tiae imaging device at 
two different distances. 
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Figure 9B is an illustration of how the different position of the object reflecting a 
given ray causes the ray to impinge on a different location of the imaging device (CCD 
chip). 

Figure 9C is an illustration of pixel coordinates in X and Y directions for portions of 
5 the pattern (crosses A, L, C, E, G, etc.) at a certain Z distances, with a best fit line or 
mathematical function connecting the portions together. 

Figure 10 is a illustration of an arbitrary ray R which is projected from the 
projection system onto a caUbration sxurface and captured by the electronic imaging device, 
with the catibration surface positioned at two different distances from the scanner, Zl and 
10 Z2 and separated by a distance AZ. Figure 10 illustrates a fundamental principle of the 
technique that is used for calibration of the scanner and generation of three-dimensional 
information of an object, which is considered to be an improvement over the calculations 
required by the metiiod of Figure 9. 

Figure 1 1 is an illustration of a portion of a pattern that is projected from the scanner 
15 of Figure 3 onto an object, the projection pattem comprising an array of parallel lines 
separated from each other by colored dots, it being understood that other types of projection 
patterns are possible. 

Figure 12 is an illustration showing that the various rays of the projection system 
through the pattem of Figure 1 1 can be represented by an array of N X M points. 
20 Figure 13 is a illustration of the X-Y calibration surface of Figure 8, showing the 

array of Q X P points in the calibration surface being organized into a coordinate system 
having an origin at the center of the surface and breaking the surface up into four quadrants 

i-rv. 

Figmre 14 is an illustration of the electronic imaging device, comprising an array of 
25 pixels arranged in X columns and Y rows. 

Figure 15 is an illustration showing the interpolation of pixel addresses in X and Y 
directions for given ray R from a scanned object from two points of the X — Y calibration 
plane previously imaged by the electronic imaging device during calibration. 

Figure 16 is a more detailed illustration of the location of the ray R 23 from the 
30 seamed object relative to the points of the X-Y calibration plane. 

Figures 17, 18 and 19 illustrate the pattem recognition process for captured two 
dimensional images, as a first step ui deriving three-dimensional information as to the 
object. 
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Figure 20 illustrates the points of the X-Y calibration plane from the calibration 
station of Figure 7 at two positions relative to the optical elements of the electronic imaging 
device. 

Figure 21 illustrates the relationship between the projection of ray R from the 
projection system and the X-Y calibration surface of the calibration station. 

Figure 22 is an illustration showing the relationship between the unknown distance 
Z ' of the object from the scanning device and the locations where ray R is imaged by the 
electronic imaging device at distances Zf and Z2. - - - - - 

Figure 23 is an illustration showing the coordinate system that used for the X-Y 
calibration surface in generating the entries for the caUbration tables for the scanner. 

Figure 24 is an illustration of a first calibration table for the scanner after completion 
of the Z calibration step; 

Figure 25 is an illustration of a second calibration table for the scanner after 
completion of the X-Y calibration step. The entries in the second calibration table of Figure 
25 are used to complete the mm entries in the calibration table of Figure 24. 

Figure 26 is an illustration of the first calibration table for the scanner after the 
calculations from the table of Figure 25 have been performed for ray R 3 3 and the results 
entered in the first table. It will be understood that a similar calculation from calibration 
table # 2 (Figure 25) is done for each ray at both distances and the entries in mm for each 
ray are entered in calibration table # 1 (Figure 24). 

Figure 27 is an illustration of a two-dimensional bitmap image of a tooth and 
associated anatomical stmctures captured by the electronic imaging device of the scanner of 
Figure 1, 2, 3 and 4, prior to any signal processing in the scanning node or workstation. 

Figure 28 is an illustration of the image of Figure 27 after pattern recognition and 
filtering operations have been performed. 

Figure 29 is an illustration of a single "frame" of data, that is, a three-dimensional 
point cloud of a scanned object which has been calculated from a single two dimensional 
image by the pattern recognition, decoding, and 3-D calculations described herein. 

Figmre 30 is an illustration of the points of the cloud of Figure 29 in which three 
adjacent points of the cloud are joined together to form triangle surfaces. 

Figure 31 is a view of the three-dimensional surface formed from the triangle 
surfaces shown in Figure 30. 
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Figure 32 is a view of the surface of Figure 31, smoothed by a smoothing algorithm 
to give a smoother representation of the surface of the object 

Figure 33 is another example of a bitmap image obtained by the electronic imaging 
device of the scanner. 

5 Figure 34--is a plan view of the three-dimensional surface obtiained from flie two- 

dimensional bitmap image of Figure 33. 

Figure 35 is a perspective view of the three-dimensional surface shown in figure 34. 
Figure 36 is ~a flow chart shown illustratiag the steps performed to generate a 
complete three-dimensional model of the dentition of a patient from a series of scans of the 
1 0 upper aad lower j aws. 

Figure 37A is an illustration of a mathematical model of a surface of an object after 
generation of several frames of data and registration of such frames relative to each other to 
generate a complete model of the surface of the object. 

Figure 37B is a illustration of one small three-dimensional section of the model of 
15 Figure 38 A, showing points from three different frames that are located in approximately 
the same position in three dimensional space after the registration of these frames has been 
completed. 

Figure 37C is a screen shot on the user interface of a back office work station of 
Figure 1, showing triangle surfaces for points comprising one frame obtained from scanning 
20 a tooth. 

Figure 37D is a screen shot showing a registration of a large number of overlapping 
frames on the tooth, of which the frame of Figure 37C is one frame. Figure 37D illustrates 
that the low resolution of the scanner's projection pattern, as indicated by the widely spaced 
points in Figure 37C, is compensated by registration of large overlap of frames, as 
25 illustrated hi Figure 37C, and results in a high resolution surface. 

Figures 38A-38C are an illustration of a two-dimensional cross-coiTelation 
procedure in the X and Y directions. The procedure, along with the procedmre in. Figures 
37A aad 37B, is used to find an initial entry poiat into a registration algorithm between 
successive frames of data. 
30 Figure 39 A and 39B are an illustration of a one-dimensional correlation procedure in 

the Z direction for two succesisive frames. 

Figures 40A-40D are an illustration of a frame to frame registration process for a set 
of frames, each frame consisting of a three-dimensional point cloud of a scanned object, 
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Each frame is typically generated from a different spatial orientation of the scanner relative 
to the object due to movement of the scanner during image capture, hence the frames 
overlap to at least some extent. The registration process is used to find a best fit between 
the frames relative to each other, and thereby provide a complete three-dimensional virtual 
model of the surface of the object from all of the frames. 

Figure 41 illustrates the normal vectors used in the process of Figure 40. 
Figure 42 illustrates the summation of the normal vectors from frame 1 to reach a 
net normal vector N net- 
Figure 43 illustrates the vectors Vi and Ni from the process of Figure 40; 
Figiure 44 illustrates the cross product of vectors Vi and Ni; 

Figure 45 illustrates the parameter R, which is used in the fi^e regisfration process 
to discard or filter out points which are greater than a distance R from the triangle surfeces. 

Figure 46 is an illustration of the closeness factor or quality index, measured as die 
magnitude of the net noimal vector as a ftmction of iterations of the process of Figure 
40, showing how the quality index improves with successive iterations of the registration 
process. 

Figures 47 A and 47B are an illustration of a cumulative registration procedure, 
which is an alternative to the frame to frame registration procedure of Figure 40. 

Figure 4SA-48C are a flow diagram of a cumulative registration process. 

Figure 49 is an illustration of a set of frames illustrating a different order for fi^me 
registration than frame to frame registration, with the order based on the location on the 
surface of the object for a given firame relative to the location on the surface for other 
frames. 

Figure 50 is a simpKfied illustration of a set of frames, showing the order in which 
the frames were obtained, with the neighborliness of the frames relative to other fi^es 
being the basis for the registration order shown in Figure 49. 

Figure 51 is another illustration of a set of frames, with registration of frames 
performed in accordance with the method of Figure 49, with the marking in frames 2, 3 6 
and 7 etc. indicating that that frame has been registered. The marking is just a way of 
illustrating that the computer keeps track of which frames have not been registered, as a 
check to insure that no frames are omitted during the regisfration procedure of Figure 49. 

Figure 52 is an illusfration of cumulative registration based on the first captured 
firame (Fl) as being the base line for all successive registrations. 
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Figure 53 illustrates an alternative registration procedure in which each frame in the 
set of frames is registered to a cumulative registration 3-dimensional model of the object, in 
sequential order, with one iteration of the frame registration process. This is followed by an 
updating of the cumulative 3-dimensional model and a r^eat of the registration process 
5 with updated values for the transformation matrix [T] for each frame. 

Figure 54 is a screen shot of a workstation computer (either scanning station or back 
office server workstation), showing the available registration parameters and variables that 
can be changed by the user when performing either a frame to frame registration or a 
cumulative registration. 

10 Figure 55 is a screen shot from a workstation computer showing a frame to frame 

registration in accordance with Figiure 40 for two frames in a set of frames. 

Figure 56 is a screen shot showing the results after forty five iterations of Ihe frame 
to frame registration process of Figure 40, and with the right hand side of the screen shot 
showing tiie two frames superimposed on each other. 

15 Figure 57 is a screen shot showing a graphical representation of a three-dimensional 

model of a patient's upper front teeth after a frame to frame registration. The user is 
applyiug landmarks to the teeth as a preliminary step in treatment planning, and as a step in 
registering overlapping segments of a scanned upper jaAV relative to each other to calculate a 
complete model of the upper jaw and associated dentition. 

20 Figure 58A-58F are a series of illustrations showing the generation of an individual 

tooth model from a scanned tooth, shown in Figure 5 8 A, and a template tooth, shown in 
figure 58B. A library of template teeth similar to Figure 58A are stored as three- 
dimensional computer models in computer memory. The individual tooth model is a three- 
dimensional tooth object having a single set of points defining the boundaries of the tooth. 

25 The individual tooth model reduces the amount of data required to represent the tooth, as 
compared to the data rq)resenting the tooth after a cumulative registration of a large number 
of frames. Individual tooth models are also invaluable in interactive orthodontic treatment 
planning since they can be independentiy moyed relative to each other in simulation of 
treatment scenarios, 

30 Figure 59 is an illustration of the tooth model of Figure 5SD positioned in the 

computer model of the patient's dentition, surrounded by other anatomical structures. 

Figure 60 is a screen shot from an orthodontic workstation showing the computer 
model of the patient's teeth positioned in a target or desired condition, as a result of the user 
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selecting an archfoim for the patient and the computer placing the teeth along the arch 
selected by the user. Figure 60 also shows the various parameters by which the orthodontist 
can adjust the shape of the arch, the distance between the teeth, the distance between the 
molars, and other parameters, so as to provide a unique and customized target situation for 
the patient. 

Figure 61 is another screen shot showing the computer model of the patient's teeth 
in a target situation, also showing the numerous parameters available to the orthodontist to 
customize the tooth position, orientation, angulation, torque, and other parameters on a tooth 
by tooth basis for the target archform. 

Figure 62 is another screen shot showing a front view of the target situation and 
additional parameters available to the orthodontist for moving teeth relative to each other in 
plaoning treatment for the patient 

Figure 63 is a screen shot of a target situation for the patient showing the vhtual 
tooth in a target position, a set of virtual brackets placed on the teeth, and a virtiial archwire. 
15 Figures 64A-64D are four views of a virtual model of a portion of the dentition 

generated by the scanning system of figure 1, illusti-ating alternative metiiods for separating 
teeth from associated anatomical sti^ctiire, e.g., other teeth and gingival tissue, as a part of 
the process described in conjunction with Figure 58A-58F. 

Figure 65 is an illustration of an interpolation procedure that can be used in the 
process described in conjunction with Figures 58A-58F to fill in holes in scan data from a 
template object. 

Figure 66 is a screen shot of a workstation user interface display showing a patient 
records screen which the user access various information regardmg the patient, including 
dental history. X-ray photographs, medical history, photographs of the face, teeth and head 
25 and the three-dimensional model of the malocclusion. 

Figure 67 is an illustiration of a series of icons that appear on a screen display that 
provide some tools for viewing the three-dimensional model of the patient's dentition. 

Figure 68 is an illustration of a set of icons which are part of the screen displays, 
which act as a navigational tool and allow the user to manipulate the three-dimensional 
30 models of teeth and brackets on the display. 

Figure 69 is a screen shot from the tireahnent planning software showing a set of 
mdividual tooth objects representing the observed stage of a patient suffering from a 
malocclusion. 
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Figure 70 is another screen shot from the treatment planning software, showing the 
observed stage and the placement of virtual three-dimensional brackets on the surfaces of 
the teeth. 

Figure 71 is another screen shot from the treatment planning software, sho^ving 
5 several views of the observed stage and the fields by which an orthodontist can enter values 
to alter the relation of the upper jaw to the lower jaw as an initial step of planning treatment. 

Figure 72 is another screen shot showing several views of the malocclusion 
displayed simultaneously, similar to Figure 71. 

Figure 73 is a screen show showing a cross-section or "clipping plane" view through 
10 the upper arch in a target situation. 

Figure 74 is a screen shot illustrating of a portion of a target arch, showing a vertical 
cross-section or clipping plane taken flirough the teeth. This view is helpftil in adjusting the 
relation between the iq)per and lower jaw. 

Figure 75 is a screen shot shovmig the placement of the virtual brackets on the teeth 
15 at the malocclusion, showing the user clicking on an icon to establish an initial archfoim for 
the upper arch. 

Figure 76 is a screen shot shovnng the computer model of the patient's teeth 
positioned in a target or desired stage, as a result of the user selecting an archform for the 
patient and the computer placing the teeth along the arch selected by the user. Figin-e 76 
20 also shows the various parameters by which the orthodontist can adjust the shape of the 
arch, the distance between the teeth, the distance between the molars, and other parameters, 
so as to provide a unique and customized archform for the patient. 

Figure 77 is another screen shot showing the computer model of the patient's teeth 
in a target stage, also the brackets and the orthodontic archwire, and showing the numerous 
25 parameters available to the orthodontist to customize the tooth position, orientation, 
angulation, torque, and other parameters on a tooth by tootti basis for the target archform. 

Figure 78 is another screen shot showing a view of the target situation, with brackets 
and archwire, showing fields allowing the orthodontist to moving the teeth objects relative 
to each other in planning treatment for the patient. 
30 Figure 79 is a screen show showing a bracket offset correction being entered to 

move tooth number 16 into an improved occlusal relationship with the opposing jaw. 

Figure 80 is a screen shot showing the tooth movement that occurs with tooth 
number 16 when the bracket offset correction is made. 



22 



WO01/807r.l PCT/USOl/11969 

Figure 81 is another screen show showing the target stage, with the brackets and 
archwire, showing a tooth moved in the buccal and coronal directions by an amount 
mdicated by the orthodontist, and the correction incorporated into the archwire. 

Figure 82 is another screen shot showing a space management feature by which the 
target situation can be adjusted by specifying spaces between teeth or by extraction of teeth. 

Figure 83 is a screen shot illustrating the simulation of an extraction of a tooth 
number 41 . 

- Figure 84 is a screen shot showing the user modifying the mesial gap between the 
teeth and showing how the user-specified mesial gap is instantaneously represented on the 
screen. 

Figure 85 is a screen, shot showing the user modifying tooth position in a target stage 
on a tooth by tooth basis using a Isonding correction feature. 

Figure 86 is a screen shot showing a wire tab, which allows the user to make 
changes in the shi^je of ah archwire without changing bracket position. 

Figure 87 is a screen shot showing a wire offset tab, which allows the user to change 
thte the bend size, location, etc., in the wire. 

Figure 88 is a schematic representation of an archwire manufacturing system shown 
in Figure 1. 

Figure 88A is a.block diagram of the systan of Figure 88. 

Figure 89 is a perspective view of a moveable robot arm used in the manufacturing 
system in Figure 88; in Figure 89 the gripping tool at the distal end of the aim is omitted for 
sake of clarity to show the various other aspects of the arm. 

Figure 90 is perspective view of the robot arm of Figure 89, showmg the movement 
of one of the arm joints and the corresponding motion of the arm. 

Figure 91 is a detailed perspective view of the gripping tool that is mounted to the 
distal end of die moveable robot arm of Figure 89 and Figure 90. 

Figure 92 is a perspective view of the fixed gripping tool of Figure 88 and the 
grippmg tool of Figure 91, in which an orthodontic archwire is gripped by the tools. 

Figure 93 is a perspective view of a magazine of Figure 88 that holds a plurality of 
straight archwires. 

Figure 94 is a detailed perspective view of the moveable gripping tool grasping one 
of the archwires fiom the magazine of Figure 93. 

Figure 95 is a perspective view of an alternative arrangement of a moveable robot 
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arm- 
Figures 96A and 96B are perspective views of alternative magazine constructions to 
the magazine of Figure 93. 

Figure'97 is a schematic illustration of a conveyor system that carries archwires to 
5 the robot arm of Figure 88- 

Figure 98 is a diagram illustrating the robot software as it relates to the production 
flow in producing orthodontic archwires. 

* Figure 99 is a simphfied illustration of a set of teeth showing the origin of a 
coordinate system that is used to calculate bracket location for a set of brackets, in three 
10 dimensions, for a patient. The bracket location for the teeth in a target situation determines 
the shape of an orthodontic archwire. 

Figure 100 is an illustration showing the vectors drawn from the origin of the 
coordinate system to the center of the brackets. 

Figure 101 is a perspective view of an orthodontic bracket. 
15 Figure 102 is an illustration of a vector drawn from the origin of the coordinate 

system to the bracket, a normal vector N perpendicular to the slot surface of the bracket, and 
a tangential vector T extending in the direction of the slot of the bracket. 

Figure 103 shows the normal vector Y for a particular bracket, the tangential vector 
X, the tangential distance and antitangential distance AT^. 
20 Figure 104 shows in matrix form the values for an individual bracket which describe 

the location of the bracket and its orientation, which are used to g^erate the commands for 
the robot to form the orthodontic archwire. 

Figure 105 is an illustration of a set of points PI, P2, P3, . . . PN which represent a 
set of bending points associated with individual brackets for a patient in a target situation. 
25 The location of the points in the three-dimensional coordinate system is known. 

Figure 106 is an illustration of a section of wire between points PI and P4 in which a 
bend is placed between points P2 and P3. 

Figure 106 A is an illustration of four points and a curve defined by a Bezier spline, a 
technique used to calculate the shape of the bend in the wire between points P2 and P3 in 
30 Figure 106. 

Figure 106B is a flow chart of an algorithm to calculate the Bezier spline of Figure 
106 A and the length of the curve. 

Figure 107 is a graph of force as a function of deflection for a worlipiece such as a 
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wire. The graph illustrates that that when a certain amount of force, Fl, is appUed to the 
worlq>iece Ld then released, a deflection D2 results. When the force is released, the 
amount of remaining deflection, Dl, is less than the deflection observed when the force is 
appHed to the wire, D2, since the wire has elastic properties. 
5 Figure 1 08 is an iUustration of the stresses found in a wire when it is bent. 

Figure 109 is an elevational view of the gripping fingers of the fixed gripping tool of 
Figure 92, showing the origin of a coordinate system used by the robot in bending wire. 

Figure 1 1 0 is a top view of the gripping fingers of Figure 109. 

Figure 1 1 1 is flowchart illustrating a defoimation-controlled overbending procedure, 
10 which may be used to compensate for the elastic properties of the wire demonstrated by 
Figure 107. 

Figure 112 is an illustration showing the overbending niethod set forth in Figure 

111. 

Figures 113A-113E are a series of schematic drawings of the fixed and moveable 
15 gripping tools of Figure 92, showing how they moved relative to each other and grip and 
release the archwire to place the archwire in position to form a bend between points P2 and 
P3 of Figure 105. 

Figure 1 14 is a schematic illustration showing how the movable gripping tool bends 
an archwire while maintaining a constant distance firom the fixed gripping tool. 
20 Figures 115A-115C illustrate how a bend may be formed in a series of steps. 

Figures 116A-116D are a series of schematic drawings of the fixed and moveable 
gripping tools of Figure 92, showing how they move relative to each other to place the 
archwire in position to form a bend between points P4 and P5. 

Figure 117 is an illustration of the points defining a portion of an archwire, and 
25 illustrating a technique for placing bends in wires where a substantial distance exists 
between the straight wire segments. 

Figure 118 is an illustration of a portion of an archwire showing a bend fonned 
therein to increase the forces applied by the wire when the wire has nearly straightened out, 
e.g., near the end of treatment. 
30 Figure 1 19 shows the wire segment of Figure 1 1 8 instaUed between two teeth. 

Figures 120 shows a wire with a loop in the wire. 
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Figures 121 A.121B illustrate one possible method of fonning the loop in the wire of 
Figure 120. 
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DETAILED DESCRIPTION OF PREFERRED EMBODIMENT 
Part 1. Overview 

Figure 1 is an illustration of an orthodontic care system 10 incorporating a scanner 
system 12 in accordance with a representative embodiment of the invention. The scanner 
5 system 12 includes a hand-held scanner 14 that is used by the orthodontist or his assistant to 
acquire three-dimensional information of the dentition and associated anatomical structures 
of a patient. The images are processed in a scanning node or workstation 16 having a 
central processing unit, such as a general-purpose computer. The scanning node 16, either 
alone or in combination with a back-office server 28, generates a three-dimensional 
10 computer model 18 of the dentition and provides the orthodontist with a base of infomiation 
for diagnosis, planning treatment, and monitoring care for the patient. The model 18 is 
displayed to the user on a monitor 20 connected to tiie scaiming node 16. 

As noted above, the scanner system 12 described in detail herein is optimized for in- 
vivo scanning of teeth, or alternatively, scanning a plaster model/ of tiie teeth and/or an 
15 impression of the teeth. However, it will be apparent to persons skilled in the art that the 
scanning system 12 can by readily optimized for a variety of other diagnostic and/or 
treatment planning and'or monitoring uses in the medical arena. An example is scaiming 
tiie face or head and planning plastic or orthopedic surgery. It can be readily adapted to 
virtually limitless number of applications m industrial, manufacturing, forensic, 
20 archeological, scientific, archival or other applications. 

The orthodontic care system consists of a plurality of orthodontic chnics 22 which 
are linked via tiie Intemet or other suitable communications medium 24 (such as the public 
switched telephone network, cable network, etc.) to a precision appliance service center 26. 
Each clinic 22 has a back office server work station 28 having its own user interface, 
25 mcluding a monitor 30. The back office server 28 executes an orthodontic treatment 
planning software program. The software obtains the three-dimensional digital data of the 
patient's teeth from the scanning node 16 and displays the model 18 for the orthodontist. 
The treatment planning software includes features to enable the orthodontist to manipulate 
the model 18 to plan treatment for the patient. For example, the orthodontist can. select an 
30 archform for the teeth and manipulate individual tooth positions relative to the archfonn to 
arrive at a desired or target situation for the patient. The software moves the virtual teeth in 
accordance with the selections of the orthodontist. The software also allows the 
orthodontist to selectively place virtual brackets on the tooth models and design a 
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customized archwire for the patient given the selected bracket positions. When the 
orthodontist has finished designing the orthodontic appliance for the patient, digital 
information regarding the patient, the malocclusion, and a desired treatment plan for the 
patient are sent over the communications medium to the appUance service center 26. A 
customized orthodontic archwire and a device for placement of the brackets on the teeth at 
the selected location is manufactured at the service center and shipped to the clinic 22. 
Tlie invention is also applicable to other types of appHance systems; brackets and archwires 
are shown in the illustrated embodiment but other types of appliance systems can benefit 
firom the scanning system described herein, such as removable aUgning devices; retainers, 
Herbst appliances, etc. 

As shown in Figure 1. the precision appHance service center 26 includes a central 
server 32, an archwire manufacturing system 34 and a bracket placement manufacturing 
system 36. These details are described later. 

Figure 2 is a more detailed block-diagram of the scanning system 12, suitable for use 
in the orthodontic care system of Figure 1. The scanning system 12 is a mechanism for 
capturing three-dimensional information of an object 40, which in the present example is the 
dentition and surrounding anatomical structures of a human patient, e.g., gums, bone and/or 
soft tissue. The scanning system 12 includes a scanner 14 which is used for image capture, 
and a processing system, which in the illustrated example consists of the main memory 42 
and central processing unit 44 of the scanning node or workstation 16. 

The scanner 14 includes a projection system 46 that projects a pattern onto the object 
40 along a first projection axis 48. The projected pattern is formed on a sUde 50 which is 
placed in &ont of a Kght source 53. hi the iUustrated embodiment, the light source 53 
comprises the terminus of a fiber-optic cable 51. The cable 51 carries a high intensity flash 
generated by a flash lamp 52 located in a base unit 54 for the scanner. A suitable flash lamp 
is the model FX- 1 160 flash unit available from Perkin Elmer. The illuminations of the 
flash lamp 52 cause the pattern contained in the shde 50 to be projected onto the three- 
dimensional surface of the object. Further details on the types of patterns suitable for the 
pattem are set forth in the following co-pending patent applications of Rudger Rubbert et 
al:, serial no. 09/254,755 filed March 9, 1999; serial no. 09/560, 13 Ifiled April 28, 2000, and 
serial no. 09/673,863 filed November 30, 2000 assigned to the assignee of the present 
mvention, the contents of which are incorporated by reference herein. A presently preferred 
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projection pattern is described below. The details on the optics of the projection system 46 
are set forth in further detail below. 

The scanner 14 further includes an electronic imaging device 56 comprising an array 
of photo-sensitive pixels. A preferred embodiment is an off-the-shelf; color-sensitive, 
5 charged- coupled device (CCD) of a size of 1,028 X 1,028 pixels arranged in an airay of 
rows and columns. The Sony ICX205AK CCD chip is a suitable electronic imaging device. 
The electronic imaging device 56 is oriented perpendicular to a second imaging axis 58, 
which is off-set from" the projection axis 48. The angle *F between the projection and 
imaging axes need not be known in a preferred embodiment of the invention. However, if 
10 the 3D calculations are made in accordance with the parameters of Figure 9, then the angle 
and the separation distance between the center of the imaging device 56 and the center of 
the light source 53 need to be known. 

The angle T will be optimized during design and manufacture of the scanner 
depending on the desired resolution required by the scanner. This, in turn, is dependent on 
-15 the degree to which the surface under scrutiny has undercuts and shadowing features which 
would result in the failure of the imaging device to detect the projection pattern. The greater 
the angle ^, the greater the accuracy of the scanner. However, as angle *P increases, the 
presence of undercuts and shadowing features will block the reflected pattern and prevent 
capture of the pattern and subsequent three-dimensional analysis of those portions of the 
20 surface. Angle is shown somewhat exaggerated in Figure 2, and will generally range 
between 10 and 30 degrees for most applications. 

The electronic imaging device 56 forms an image of the projection pattern after 
reflection of the pattern off of the surface of the object 40. The reflected patterns imaged 
by the imaging device contain three-dimensional information as to Uie surface of the object, 
25 and this information needs to be extracted from the images. The scanning system therefore 
Includes a processing subsystem which is used to extract this information and construct a 
three-dimensional virtual model of the object 40. In the preferred embodiment, this 
processing subsystem consists of a memory 42 storing calibration information for the 
scanner, and at least one processing unit, such as the central processing unit 44 of the 
30 scanning workstation 16. The location of the memory and the processing unit is not 
important. They can be incorporated into the scanner 14 per se. Alternatively, all 
processing of the images can take place in the back office server 28 or in anotha: computer. 
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Alternatively, two or more processing units could share the processing in order to reduce the 
amount of time required to generate the three-dimensional information. 

The memory 42 stores a calibration relationship for the scanner 14. The cahbration 
relationship, which can be in the form of a table or one more mathematical functions, 
comprises information used to compute three-dimensional coordinates of points on the 
object that reflected the projection pattern onto the imaging device. The information for the 
table is obtained during a calibration step, performed at the time of manufacture of the 
- scanner 14.- The cahbration table includes an array of data storage locations that contain 
two pieces of infonnation. Firstly, the calibration table stores pixel coordinates in X and Y 
directions for numerous portions of the projection pattern that are imaged by the electronic 
imaging device 56, when the pattern is projected onto a calibration surfece at two different . 
distances during a cahbration procedure. Secondly, the table stores distance information, 
(e.g., in units of tenths of milhmeters), in X and Y directions, for the portions of the 
projection pattern;. imaged at the two different distances. A preferred method for generation 
and use of the cahbration table is explained in further detail below. 

The scanning system requhres at least one processing unit to perform image 
processing, tinree-dhnensional calculations for each image, and registration of frames to 
each other. The processing unit 44 in the illustrated embodiment is the central processing 
miit (CPU) of tiie scanning work station 16. The CPU 44 processes the image of the pattern- 
after reflection of the pattern off the surface of the object 40 and compares data from the 
image to the entries in the cahbration table. From that comparison (or. more precisely, 
interpolation relative to the entries in the table, as explained below), the processing unit 44 
derives spatial information, in three dimensions, of points on flie object tiiat reflect the 
projected pattern onto the electronic imaging device. 

Basically, during operation of the scanner to scan an object of unknown surface 
configuration, hmrdreds or thousands of images are generated of the projection pattern as 
reflected off of tiie object in rapid succession as the seamier and object are moved relative to 
each other. For each image, pixel locations for specific portions, i.e., points, of the reflected 
pattern are compared to entries in the cahbration table. X, Y and Z coordinates (i.e., three 
dimensional coordinates) are obtained for each of these specific portions of the reflected 
pattern. For each picture, the sum total of aU of fliese X, Y and Z coordinates for specific 
points m the reflected pattern comprise a three-dimensional "frame" or virtual model of the 
object When hundreds or thousands of images of the object are obtained from different 
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perspectives, as the scanner is moved relative to the object, the system generates hundreds 
or thousands of these frames. These frames are then registered to each other to thereby 
generate a complete and highly accurate three-dimensional model of the object 40. 

Stray data points are preferably canceled out in generating the caUbration table or 
5 using the calibration table to calculate three-dimensional coordinates. For example, a 
smoothing function such as a spline can be calculated when generating tiie entiies for the 
calibration table, and tiie spline used to cancel or ignore data points tiiat deviate 
significantly from the spline. 

Figure 2 also shows a few other features of the presently preferred scanning system 
10 12. After the CCD imaging device 56 captures a single image, the analog voltage signals 
from flie device 56 are amplified in an ampUfier 57 and fed along a conductor 59 to an 
analog to digital converter 60. The digital signal is converted into a bitinap sfream of digital 
image data. The data is formatted by a module 61 into an IEEE 1394 "firewire" format for 
transmission over a second conductor 62 to the main memory 42 of the scanner work station 
15 16. The scanning system includes an optical scanner holder 64 for the user to place the 
scanner after the scanning of tiie dentition is complete. These details are not particularly 
important and can vary considerably from the illusti^ted embodiment. As noted earher, 
preferably the scanning system is constructed to provide a minimum of equipment and 
clutter at the chair side. Hence, the scanning work station 16 is preferably located some 
20 distance away frpm the chair where the patient sits. The cable leading from the scanner 14 
to the base station and/or workstation 16 could be suspended from the ceiling to further 
eliminate chairside clutter. 

The scanning work station 16 also includes tiie monitor 20 for displaying the 
scanning results as a tiiree-dimensional model 18 of the dentition in real time as the 
25 scanning is occurring. The user interfece also includes a keyboard and mouse for 
manipulating the virtual model of the object, and for entering or changing parameters for tiie 
scanning, identifying sections or segments of scans tiiat have been obtained, and other 
featiires. The scanning station may also include a foot switch, not shown, for sending a 
signal to the CPU 44 indicating tiiat scanning is commencing and scanning has been 
30 completed. The base station may alternatively include a voice recognition module tiaat is 
trained to recognize a small set of voice commands such as START, STOP, AGAIN, 
REPEAT, SEGMENT, ONE, TWO, THREE, FOUR, etc., tiiereby eliminating tiie need for 
tiie foot switch. Scanner start and stop commands from tiie CPU 44, in tiie form of contirol 
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signals, are sent to the Ught source Sa, thereby controlUng the illumination of the lamp 52 
during scanning. 

The light source 52 operates at a suitable ftequency, preferably at least areater than 
one flash per second, such as six flashes per second, and the frame rate of the CCD imaging 
device 56 IS synchronized with the flash rate. With a fi^.e rate of 6 frames per second, and 
a scamung motion of say 1-2 centimeters per second, a large of overlap between hnages is 
obtamed. The size of the mirror at the tip 68 of the scanner influences the speed at which 
scammig is possible. The illusfrated embodiment-of the mirror at the tip 68 is 18 mm 
square. A larger mixxor reflects more surface of the object and enables faster scamiing. A 
smaller mnror requires slower scamnng. Tie larger the mirror, the more difficult in-vivo 
scammig becomes, so some trade-off between size and utility for in-vivo scamiing exists. 

This overlap between images generated by the scanner 14, and resulting three 
dmiensional frames, aUows a smooth and accurate registration of frames relative to each 
other. The frame rate and permissible rate of seamier motion wiU depend on many factors 
and can of course vary within the scope of the invention. Flashing a high intensity flash 
lamp for a brief period of time is a preferred embodiment since it is desirable to reduce the 
exposure tune of the CCD imaging device 56 to reduce blurring since relative motion exists 
between the scanner and the object. A high intensity lamp is desirable to achieve sufficient 
signal strength from the hnaging device. A preferred embodhnent uses 5 ^sec flash times 
with similar exposure periods. An alternative embodiment would use a constant 
lUmmnation source of high intensity, and control exposure of the imaging device using a 
shutter, either a physical shutter or usmg elecfronic shutter techniques, such as draining 
charge accumulating in the pixels prior to generating an image. Scamiing using longer 
exposures would be possible without image blur, using electronic image motion 
compensatio9:techniques described in Lareau, et al., U.S. Patent 5,155,597. 

Figi^e 3 is a perspective view of a hand-held seamier 14 used to acquire mformatioa 
of an object under scmtiny, suitable for use in the orthodontic care system of Figure 1 The 
projection system 46 and the elecfronic imaging device 56 of Figure 2 are contained in the 
housmg 65 for the seamier. The housing 65 is sized and shq,ed to be held in a hmnan hand. 
The scanner 14 includes an elongate distal portion 66 having a tip 68. The tip 68 is sized 
and shaped such feat it can be inserted into and moved within an oral cavity of a hmnan so 
as to enable scamiing of anatomical sfructures mside fee oral cavity. A heatmg mirror (not 
shown) is placed on fee underside of fee tip 68 to direct fee projection pattern from the 
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optics of the scanner onto the object and to direct the reflected pattern from the object 
towards the imaging optics 108 of Figure 7 associated with the electronic imaging device 
56. The miiror housing has AJC conductive heatmg coil that heats the mirror. The mirror 
is heated to approximately 40 degrees to prevent fogging of the miiror while the scanner is 
used in-vivo. 

Figure 4 is an illustration of a patient 70 being scanned with the hand-held scanner 
14 of Figure 3. The checks and hps are retracted from the teeth and the tip 68 of the scanner 
is moved over all the surfaces of the teeth in a sweeping motion at a velocity of periiaps 1-2 
centimeters per second. The entire upper or lower jaw may need to be scanned in a series of 
scans, one for the left side, one for the right side, and one for the front. These individual 
scans are registered to each other as described below. Voice commands or activation of the 
foot switch (not shown) indicates when each scanning segment is initiated and terminated. 
The entire process takes just a few minutes. Depending on the color and translucency of the 
object and the illumination intensity and frequency of the Hght source in the scanner, it may 
be necessary to apply a very thin coating of a bright reflective substance such as Titanium 

Dioxide to the object 

While Figure 4 illustrates in-vivo scanning of a human patient, the scgnrier can of 
course be used to scan a plaster model of the dentition if that is preferred, or an impression 
taken from the patient. When scanning an impression or a plaster model, the scanning may 
be formed in a single pass, without the need for registering scan segments to each other. It 
is also possible that a scan of a patient may be partially taken in vivo and the remainder 
from a model or an impression. 

Figure 5 is a block diagram of the back office server of Figure 1 showmg the 
elements used to calculate the digital model of the patient' s dentition. After the scanning 
workstation has processed all the images captured by the scanner and generated a set of 
tiiree dimensional fi^es, the frame data is transmitted to tiie back office server 28. The 
back office server 28 performs a cumulative registration process for the frames and ultimaty 
generates and displays the digital model on a screen display 30. The raw scanner data in 
the form of three-dimensional frames is stored in tiie main computer memory 72. The 
frame data for N captiired images, i = 1 ... N from the scanner is stored in the hard disk 
74. The hard disk also stores a set of (N -1) transformation matrices [T] ; , for 1 = 2 - N. 
The transforaiation matrices basicaUy contain information as to how each frame of three- 
dimensional points needs to be translated and rotated in a three-axis Cartesian coordinate 

33 



10 



15 



WO 01/80761 

PCT/USO 1/11969 

system in order to be registered with the other frames in a best-fit manner. One of the 
frames, such as the first frame in the series/is a starting point for registration and no 
transformation matrix is obtained for that frame. The generation of the transformation 
matrices, and use . of the matrices with the frame data to generate the three dimensional 
5 model, is -described in finther detail below. 

2. Three-Di mensional Tmase Generarinn 
" ~ "^'^ general- introduction-and overview in mind,-a presently preferred 

process of capturing two dimensional images with the scanner and generation of a three- 
dimensional model for each image will now be described in detail in this section. Figure 6 
is a flow diagram iUustrating the sequence of steps used by the processing unit 44 in the 
scanning station 16 (Figure 2) to calculate three-dimensional information of an object for a 
single image captured by the scanner 14. This process shown in Figure 6 is performed for 
each image. 

This process can be executed in one processor, or may be performed or shared by 
multiple DSP processors sharing access to a memory storing the captured two-dimensional 
bitm^ images from the scanner, e.g., memory 42 of Figure 2. The type of processor is of . 
course not important. The goal of distributed processing is to reduce the time required for 
processing the scanned image. For example, one processor could perfonn the cross- 
correlation process described below, another perform the pattern recognition, another 
decoding, and another 3-D calculation. These processes could be performed independently, 
each process ^associated with independent files shared on a disk memory. The processor 
^'^Sned ^. .o^e or more of the processes of Figure 6 accesses the ffles as needed. For 

decodiiig process is AA^tten on a disk or memory avaUable 
to the oth^; pi^c^iig' units. The processing units are on a network in this example. The 
processor:^sig£ed to '3-D calculation access the decoding output file and writes the result, a 
- set of . coordinates for N frames and N-1 transformation matrices, onto the disk or memoiy. 
Any one of the processors assigned to perform registration could then access the N fi^e 
data and.the N^l fransformation matrices and perform the registration procedures described 
30 below. 

The J)r^ocess of 'Fijgure 6 consists of four principal steps: a pattern recognition 
process 80,' a decoding process 82, a process 84 of derivation or calculation of tfaree- 
dimensioiial^ cooi^ates for each poiat in the detected and decoded pattern, and finally a 
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Step 86 of storing the three dimensional coordinates in a memory, such as the memory 42 of 
the scanning work station 16. Again, this process is perforaxed for each captured image 
during the scanning process. In a typical scamiing scenario, hundreds or even thousands of 
images may be captured, hence this process of Figure 6 may be performed hundreds of 
times. The result of the process is the storage of a large number of sets of three-dimensional 
coordinates, each set or "frame" associated with a single c^tured image. The registration 
of these frames relative to each other to generate a complete virtual model of the object is 
described in Part 3 ofthis document. - 

As the scaimer is moved over the dentition, the imaging device acquires a series of 
bitmap im^es. The acquired bitmaps are. analyzed using pattern recognition. Pattern 
recognition detects the median lines of the projected lines, endpoints of the lines and the 
centers of the colored dots. Other types of patterns are of course possible, such as usmg 
triangles, squares, or other coding features. The coding is in the vertical direction (in the 
direction of the parallel lines), since the distortion of the projection pattem provided by the 
ID surface of the object is in this direction, as explained more ftdly in the Rubbert et al. patent 
appUcation serial no. 09/560,131 filed April 28, 2000, incorporated by reference herein. 

The pattem recognition process uses sub-pixel-precision. The color of every dot is 
analyzed as well. Based on the knowledge of the pattem structure and using the colored 
dots, the origin in the pattem for every recognized line is determined. This is necessary, as 
20 significant portions of the projected pattem may not be visible to the imaging optics due to 
shadowing, undercuts and un-sharp areas. A two-dimensional to three-dimensional 
conversion algorithm uses tiie knowledge of the origin of each imaged line with respect to 
the pattem to compute three-dimensional coordinates of the points in the object. As the 
lines are often captured only as fragments, the decoding algorithm does not always have 
25 sufficient information on each line to unequivocally assign Hiat line to the pattern. The 
algorithm therefore examine several scenarios of possible affiliations and looks for 
conflicts. In this way the inconsistent scenarios are filtered out. The lines in the projection 
pattem do not change their order in the image. For example, if the lines in the projection 
pattem are sequentially numbered 1-80, and line 43 is to the left of line 44, in the captured 
30 image line 43 wiU be always be to the left of line 44 and never to the right of Ime 44. 
Inconsistent scenarios are indicated where the order of lines is violated. The correct order of 
the lines can be deduced by a suitable algorithm that examines the scenarios based on line 
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order and eliminates aU those where conflicts or inconsistent line numberings exists. A 
unique solution will be found. 

While the preferred embodiment of the three^ensional conversion algohthm is 
. based on a sophi^lieated caHbl^tion process and does not make use of any knowledge of U,e 
opttcal parameters, an alternative embodiment could us. general principle of analytical 
mangulation assmning that we do have such knowledge. A^alyUcal triangulation will be 
explamed with reference to Figure 9. The projecting device projects a pattern of 
dtstrnguishable elements onto a surface. IMs projected pattern is imaged. For each c^tured 
element, .s must be possible to teU it. origin a, the pattern. This allows us to detennine the 
angle a between the optical axis of the projection device and the one ray that has projected 
that element He location of the pixel at the CCD chip that has captured that element aUows 
us to detemmie the angle p between the optical axis of the imaging system and the one ray 
that leads ficm the projected element at the surface to the CCD pixel. Knowing those two 
angles, the angles of the two optical axes with respect to the baseline and the length of the 
baselme, aUows ns to calculate the spatial position of the element relatively lx> the seaming 
devtce. Ttas calculation can be done for eveo- detected element within one c^tured image 
and leads to a plurality of three^ensional points. It is important to miderstand that as a 
result a-om this calculation we receive an undistorted, true to scale representation of the 
sutftce. While every two dimensional, hnage shows distortions due to patallax efficts, the 
ttiangulation process eliminates this effest 

The analytical tiiangulation mettod requires precise knowledge of optical 
parametets. The prefetred «nbodiment using a caUbration table for the scanner does not 
require this knowledge: 

^ Scanner Manufacturft anH CalihratioTi 

Before describing the details of the process stq,s shown in Figure 6, an illustrative 
embodnnent of the scanner 14 itself and its manner of calibration wiU be described first 

Figure 7 is a cross-sectional view of aportion of the hand-held seamier 14 of Figure 
3. showmg the optical elements of the projection and imaging aspects of the seamier. The 
seamier includes tHe fiber-optic cable 51 carrying flashes from a flash lamp in the base 
station 54 of Figure 2 to a condenser system consisting of a group of lenses 53A. Light 
from the light source 53 illuminates a pattern formed in a slide 50. Wide variation is 
possible in the choice of the pattern. A presently preferred embodiment is described 
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subsequently in conjunction with Figure 11. The pattern is directed by a projection lens 
system 102 out of the scanner to the mirror housed at the tip 68 of the scanner (Figure 3) 
and towards the object under investigation. The scanner further includes a several LED 
light sources 106 (e.g., 2 or 6), which are used to provide general illumination of the object 
during scanning to assist the user in scanning the dentition. A prism 104 surrounds the LED 
light soiirces. The axis of the projection system is shown as axis 48. 

The projection pattern is reflected off of the object, reflected by the mnror in the tip 
68 and received by an imaging lens system 108 centered about an imaging axis 58: The 
received pattem is reflected off a mirror 110 onto the CCD electronic imaging device 56. 
The CCD 56 produces a voltage signal for each pixel in the device. The level of the signal is 
an indication of the level of light impingmg on that pixel, allowing an image to be produced 
from the CCD. The signals are read out of the CCD using known circuitry and amplified. 
The amplified analog signal is collected and transmitted along conductors 59 to the base 
unit for conversion to digital form. The signal from the CCD is converted into a colored 
bitmap image in the illustrated embodiment. Color is used in the projection pattem in the 
illustrated embodiment, therefore a CCD chip is selected which can detect colors. A black 
and white system is of course also possible. 

In the illustrated embodiment, the separation distance between the light source and 
the projection pattem is not known or needed, nor is the angle between the axes 48 and 58. 
However, some non-zero angle between the axes is required in order to obtain depth 
information The angle selected will depend on the type of surface or object the scanner 
will be used for. These types of implementation details will vary considerably depending 
on tiie appUcation. Furthermore, it is possible to make the two axes 48 and 58 completely 
independent of each other by locating the projection and imaging in separate, independently 
moveable devices. This is described in more detail in the patent application of Riidger 
Rubbert et al, serial no. 09/254,843, the contents of which are incorporated by reference 
herein. The calibration procedures described herein are of particular advantage when the 
projection device and the imaging device are in two separate, independently moveable units. 

Figure 8 is a perspective view of a presently preferred scanner calibration station 120 
that is used to calibrate the scanner 14 at the time of manufacture. The purpose of the 
station 120 is to obtain data for a caUbration relationship for the scanner, such as a 
calibration table, which is stored in the memory of the scanner system. Some variation in 
the design of the calibration station is possible; the principle of operation of the calibration 
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station is that the scanner is calibrated by projecting the pattern onto a reference object of 
precisely known geometry at tvvo known distances (in the Z direction) and kncvn spatial 
extent (X and Y directions). A planar reference object is preferred, but it is theoretically 
possible to use any object of known geometry, but more computationaUy con^lex to use 
such objects. The goal of the caHbration procedure is to compile a set of information (e.g., 
m the fonn of a table) that completely caUbrates the seamier and places it in condition for 
use to scan objects of unknown surface geometry, without the need for precise knowledge of 
the-mechanical -or -optical properties of the -scamier. While the -present embodiment 
descnbes a calibration table as the result of the calibration process, the infomiation may be 
m other equivalent forms, such as mathematical relationships or formulae between pLxel 
address and distance, which are operated on to derive , distance information at the time of 
use. 

Before- discussing the presently preferred calibration device and calibration 
relationship, a discussion of the principles of the calibration invention will be set forth for 

ease of understanding. 

A 3D imaging device as disclosed in this application does initiaUy not dehver 3D 
mformation, but only 2D infomiation as the CCD chip is.a 2D imaging device. Information 
on the 3- dimension therefore Jias to be determined in an extra processing step. The 
additional information that we can use to perform such a 3D calculation is the spatial 
arrangement of the optical components, such as shown in Figure 9A. Figure 9A shows a 
schematic arrangement of components. One portion of the pattern is indicated ^vith a small 
square, and tke ray along which this portion would be projected is also displayed. Hie point, 
where this r^y would intersect with the surface of the object, is displayed as a circle. At flie 
center of this circle, the portion of the pattern would be projected onto the surfece of the 
object. One of the reflected rays will run through the imaging lens system and such be 
projected onto the surface of the CCD chip, which wiU cause a corresponding signal. In this 
figure, only the center rays are displayed, e. g. the rays that nm through the center of the 
lens systems. 

Assmning that there is precise knowledge of the geometrical arrangement of the 
components, it would be possible to precisely calculate the spatial coordinates of the part of 
the surface of the object that reflects the considered portion of the pattern. THis calculation 
IS only possible under three preconditions: 
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(i) The geometric arrangement of all components of the scanning system must be precisely 
known, 

(ii) The exact characteristics of all components must be kno\wn (true x/y-coordinates of aU 
CCD pixels, including precise knowledge of the dimensions of the pattern; and 

5 (ill) The lens systems must be 'ideal' which means that the center ray must be an ideal 
straight line. 

In mass production scenario for a scaimer, it will be almost impossible to guarantee 
these preconditions. One possible approach would be to calibrate the individual devices, 
- which means that the deviations of the characteristics from the ideal configuration are 
10 detennined and noted ("compensative caUbration"). The 3D calculation will then base on 
algorithms like described before, but will additionally take into account known deviations to 
compensate for iadividual characteristics of each device. However, this compensational 
calculation has to be set up very carefully, and errors in terms of plus/minus signs will not 
easily be detected especially when the deviations are minor. 
15 Another challenge is presented by scanning devices like disclosed in 

PCT/DE97/01797 by Rubbert, where imaging device and projection device are not 
physically connected to each other, and therefore the geometrical relationship may be 
completely unknown. 

The caUbration procedure that is described herein does not require any pre- 
20 knowledge of any dimensions of the optical and mechanical components, and thus can be 
termed "independent calibration". Furthermore, even any knowledge of the angle formed 
by the two optical axes (angle of triangulation) is not required. 

The background of this procedure can be described best by again just looking at one 
specific portion of the pattern that is being projected along a ray onto the surface to be 
25 measured like indicated in Fig; 9B. It is important to understand that this portion of the 
pattern will always be projected along this specific ray R, which is defined by the optical 
characteristics of the lens systan and the arrangement of the pattern and the lens system 
with respect to each other. If this portion of the pattern is being reflected from any surfece 
sitting in the field of view, we can be absolutely sure that this point of the surface will be- 
30 located somewhere along this ray R. However, we do not know at which position along this 
ray the point is located. To be able to determine this, the "independent caUbration" method 
basically takes samples. The first sample wiU be a surface that is located at a certain 
distance Zj from the projection device. The reflected portion of the pattern will show up at a 
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specific pixel at the CCD chip (or perhaps over several pixels depending on the size of the 
portion of the pattern). It is important to understand that every surface that is ever being hit 
by this ray at this Z-distance wiU always cause a reflection of this ray R directed to this 
pixel (we are dealing with diffuse reflection, so every point being reflected will send rays 
into many directions, regardless of the angle between ray and surface). Tins impHes that 
every time when this portion of the pattern is being reflected onto die pixel, we know 
exactly that this point of the surface is located at distance Z,. 

Knowledge-of the distance between^, and the scanner isTiotrequired as long as the 
scanner will not be used as an absolute measuring system. If we want to use the scanning 
system as an absolute measuring system, which means that we want to measure the location 
of points relative to the seamier and not only relative to each other, we would then need to 
use the Z-values with respect to the origin of the coordinate system of tiie scanner. The 
iUustrated embodimait is not an absolute measuring system, but nevertheless generates 
accurate virtual models of the object true to scale. 

During the caUbration process, we will acquire a plurality of such "samples" for 
different portions of the pattern reflected off a calibration surface at different Z- distances, 
where the relative Z-distances of these levels with respect to each other must be known. It 
will be discussed further below, how many samples typically be required to receive a 
complete calibration. The result of this sampling process is flie first caHbration relationship 
that is derived for the scanner: (1) pixel coordinates for the electronic imaging device for 
numerous portions of the pattern, said pixel coordinates associated with distance 
information from the projection system in a Z direction at at least two different Z distances. 

Having this first part of the caHbration procedure done, we can determine the Z- 
component of every part of the measured surfece that is reflecting the pattern onto the CCD 
chip. However, we do not have knowledge of the X- and Y-coordinates. To get this 
information, we need to perform the second part of the caUbration. 

Again, we will take "samples", but this time we AviU not make use of the pattern that 
is being projected onto the object during normal use of the scanner (the projection unit will 
be switched off). Rather, images are obtained of a reference object in the field of view that 
is equipped with features of a known geometry, i.e., known X-Y spatial relationship. The 
simplest implementation would be a pomt In Figure 9C, such a featiire is a pluraUty of 
cross-hairs. The featiire being projected onto the CCD chip will show up at a specific pixel 
(or several pixels depending on the size of the featirre). The X/Y-coordinates of the pixel at 
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the CCD chip are then assigaed to the X/Y-value of the location of the feature (and assigned 

to a certain Z-distance). 

It is obvious that if the feature is being moved in Z-direction, the location of the 
projection of the reference object at the CCD chip will change. We therefore have a 
dependence on the Z coordinate, which signifies that the Z-location of the feature must have 
a known reference to the Z-Iocation(s) of the surfaces that have been used in the first part of 
the calibration. For instance, such a feature could be located at Z,; other features might be 
lo"cate"d-with a-known reference to Zf. " 

The first feature being captured in this manner would serve as a reference. A certam 
X- and Y-value (in mm or inches) would be assigned to the location of this feature. If such a 
feature would be placed close to the optical axis of the imaging system, it would be 
preferable to assign X=Omm and Y=Omm to this location. If we want to use the scanning 
system as an absolute measuring system, which means that we want to measure the location 
of points relative to the scanner and not only relative to each other, we would then need to 
use the X/Y/Z-values of this feature with respect to tfae origin of the coordinate system of 
the scanner. 

During the calibration process, we will again acquire a pluraUty of such "samples" at 
different X- Y- and Z-locations, where tiie relative X-, Y- and Z-values of the locations of 
these features with respect to each other and with respect to the Z-values of the first part of 
the calibration must be known. It will be discussed fiulher below, how many samples wiU 
typically be reqviired to receive a complete calibration. 

It is important to understand that the detennined relationship between the X- and Y- 
coordinates of any feature being captured and specific pixel coordinates at the CCD chip 
exists only with respect to the Z-coordinate of the feature. A movement in Z wiU change the 
XTf -value. Therefore, during normal operation of Ae scanner, when the caUbration results 
are being used to calculate 3D coordinates, we first have to calculate the Z-coordinate of any 
point on a surface using the caUbration values acquired in part 1. and basing on these results 
we can then perfomi the XAT calculation, using the calibration results of part 2 of the 
calibration process. 

There are several options with regard to the number of "samples" to take during 
calibration and the way how tiie results may be stored. The most stiaightforward approach 
would be to collect pixel coordinates for at least two Z-levels of the projected pattern. The 
number of pixel coordinates will depend on the resolution of the pattern. The Z-levels will 
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preferably be defined mthin the depth of focus of the scamer projection lens systems and 
imaging lens systems, but close to the boundaries of this depth. Having collected pixel 
coordinates for at least two levels, would allow for interpolation of aU other Z-levels. Part 2 
of the calibration procedure could also comprise features (points) distributed evenly across 
the field of view, and. those features could again be placed at txvo different Z-levels, which 
would allow for an easy interpolation of X- and Y-values. The pixel coordinates acquired in 
both parts of the calibration process could in the simplest embodiment be stored in a table. 

However, this-straightforward approach has certain disadvantages. First of all, an 
apparatus is reqmred. Otherwise it would not be possible, to place the surfeces required for 
part 1 m a controllable manner with respect o each other, and the features being captured in 
part 2 also need to be precisely placed with respect to each other and to the calibration 
surface used in part 1. Usage of such a cahbration apparatus is not a problem within an 
industrial production environment. But if scanners need to be caUbrated for instance in an 
orthodontic office, it is not recommendable to always ship such a device to the location. 

But there is no need to caUbrate each portion of flie pattern m various Z-levels. If a 
device is used, that comprises surfaces at different Z-levels, portions of the pattern will be 
projected onto levels that are closer to the scanner, and portions will be projected onto levels 
that are lurther away. It is well possible, to interpolate also the pixel coordinates that are not 
acquired during calibration. 

Assuming that portions A and C of the pattern will be projected onto level Z,. while 
portions C and D will be projected onto level Z,, we will receive pixel coordinates for 
portion A and C assigned to Level Z, (x,. and y,, for A. x^, and y^. for C) and pixel 
coordinates for portion B and D assigned to Level Z, (x^, and for B, x„ and y,, for D). It 
is weU possible to linearly interpolate for instance x^ (which has not been acquired) from 
yB2 and y^,. In the same manner y^, could be interpolated from y^, and y^,. Another way to 
receive calibration values that have not been acquired directly would be to draw the 
acquired pixel coordinates for a certain Z-level onto a sheet of paper and then to construct a 
best-fit line (either straight or curved) through those points. If the mathematical function of 
this best-fit line is stored, the pixel coordinates can be calculated using that fimction instead 
of storing them separately. The operation of determining a best-fit Une can of course also be 
done directly in the computer. The best fit hne concept is illustrated in Figure 9C. 

This procedure would work as well fdr part 2 of the caHbration procedure where 
pixel coordinates aie being acquired for specific features askigned to X-, Y- and Z-values. 
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Again only a subset of features has to be captured at each Z-level, and the remaining values 
can be interpolated in the way described above. It would therefore also be possible to use 
just one calibration device that provides surfaces at least two Z-levels to perfonn part 1 of 
the caUbration and comprises features at those surfaces that aUow for part 2. The density of 
5 portions of the pattern, i.e., features to be captured, depends on the optical quaUty of the 
components of the scanner. We should capture at least four portions of the pattern, 
preferably close to the comers of the CCD imaging device 56 to provide a reUable 
interpolation. 

The advantage of this cahbration process is that it requires absolutely no pre- 
10 knowledge of the mechanical and optical characteristics of the scanner and automatically 
compensates for iiregularities of the optical components, this including the CCD chip and 
the pattern. It is therefore useful to cahTirate scanners that are made from cheap parts, and m 
can be used on scanners that have no known relationship between the imaging and the 
projection device. 

15 

With the foregoing discussion of the principles of the invention in mind, a 
representative embodunent of a scanner caUbration device and method will be described 
with particularity with reference to Figure 8. The presently preferred scanner caUbration 
system includes mount or holder 122 for holding the scanner fixed m position during 
20 caUbration. The holder is affixed to the top of a table 124. A calibration apparatus is 
positioned directly m front of the scanner 14. The caUbration apparatus consists of a Z- 
direction earner 126 having one portion 128 fixedly mounted to the table and a second 
portion 130 which can move back and forth in the Z direction between two different 
positions Zl and Z2. An X-direction carrier 131 is mounted to the moveable portion 130 of 
25 the Z-direction carrier. The X^Urection carrier consists a first portion 132 which is mounted 
to the moveable portion 130, and a second portion 134 wHch is moveable in the X direction 
relative to the first portion 132, as indicated. 

The X-direction carrier 131 has mounted to its upper surface 136 two calibration 
devices: (1) a smooth, planar caUbration surface 138 used for calibration of the scanner in 
30 the Z-direction, and (2) an X-Y caUbration surface 140 used for caUbration of the scanner in 
the X and Y direction. The X-direction carrier also contains a tight 142 for providing back 
illumination of the X-Y caUbration surface 140. 
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To calibrate the scanner 14. carriers 126 and 131 are moved such that the Z-direction 
calibration surface 138 is positioned in front of the scanner 14. An image is taken of the 
projection pattern reflecting off the surface with the surface 138 at some arbitrary distance 
Zl from the seamier. Then the carrier 130 is moved a distance away (A Z) to a newposition 
Z2, and a second image is taken. Pixel addresses where the specific locations of the pattern 
are imaged in the electronic imaging device are detennined and stored in a calibration table 
in a memory. The distance AZ is also knoMm precisely and stored in the scanner memory or 
•m the computer that p-^oims-the scanner calibration: 

Then, the carriers 126 and 131 are moved such that the X-Y calibration grid 140 is 
placed at the distance Zl and an image is taken. THe image is generated by activating the 
source 142, with light from the source 142 passing through numerous tiny apertures 143 in 
the calibration surface 140 and impinging on the electronic imaging device 56. (Hie pattern 
iUumination source is not used in this part of the caUbration). The carrier portion 130 is 
moved to the position Z2, and another image is generated. Using the known separation 
distance between points in the X-Y calibration grid 140, X and Y distance information for 
pomts m the pattern imaged in the fct part of the calibration procedure is computed. The 
results are stored in the calibration table. IHis process is described in further detail below. 
When the seamier calibration is finished, the scanner serial number and seamier caUbration 
table (or other representation of the caHbration relationship, such as a set of mathematical 
equations) are stored in memory in the seamier or in a computer associated with the seamier 
that processes the scarmed images. • 

An alternative configm:ation of the caUbration surfaces is shown in Figure 8A. The 
caUbration device consists of a set of reflective, planar, paraUel planes 144, 144', 144" and 
144 •", of known distance from each other and from the seamier, with ^ertures 143 spaced 
at known distances along the smfaces of the planes and at kno.vn distances from adjacent 
planar surfeces. A cross feature 145 at the center forms the origin of the caUbration surface, 
and is used.as explained below. Two of the surfaces 144, 144', 144" and 144 are used for 
the Z caUbration surface. For X and Y calibration, the backlighting iUmnination source 142 
IS activated and Ught passes through the apertures 143 onto the electronic imaging device. 
The entire two-dimensional focal plane of the electronic imaging device is caUbrated from 
an mterpolation of known points in the surfaces 144, 144', 144" and 144™, in accordance 
with teachings described in detail herein. The embodiment of Figure 8A is considered more 
cumbersome than the embodiment of Figure 8, but is offered to illusfrate that other 
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configurations for a calibration surface are possible. In fact, curved surfaces or sloping 
surfaces could even be used, but the simplest surface is a planar surface oriented directly at 
the electronic imaging device. 

Thus, in one possible alternative embodiment of the invention a calibration device is 
5 provided for a scanner projecting a pattern onto an object and receiving a reflection of the 
pattern off the object. The calibration devices comprise a calibration surface 144 receivmg 
said projected pattern comprising two or more parallel surfaces (e.g., 144 and 144") of 
known separation distaneg-and-spatiail extent and a pluraUty of point sources of light 143 
provided in the two or more parallel surfaces. As described herein the point sources of Ught 
10 are apertures which allow light to pass through the surfaces 144 from tiie hght source 142, 
but other configurations are possible. For example, the point sources of tight could be Ught 
emittmg diodes arranged in an array in the surface 144. The apertures 143 are formed m a 
precise and known spatial relationship relative to each other, such as by foimmg the holes 
with a precision high powered laser on a sheet of metal. Alternatively, instead of apertures 
15 143, black dots could be formed on paper usmg a highly accurate printing process, and the 
black dots imaged by the CCD 56., 

The calibration procedure described herein represents an alternative, and more 
preferred way of computing three-dimensional information for images as compared to pnor 
art metiiods. Figure 9 is an illustration of the relevant parameters that can be used to 
20 calculate surface configuration of the object in accordance witii a known fashion. The 
method of Figure 9 requires knowledge of the separation distance D or baseline between the 
detector and the light source, the angle between the axes 48 and 58, and the angles a and p 
shown in the Figure. The present caHbration method and method of calculation of three- 
dunensional information does not require any of this information. The calibration 
25 procedure compensates for unperfections in the optics in the optical paths, and therefore 
ehminates the need for high precision optics. Further, there is no need for precise 
knowledge of the placement of the scanner relative to the cahbration planes. There is no 
need to know the angle between the axes 48 and 58, the separation distance between the 
scanner and the object being scanned, or any absolute value of the location of the object m 
30 any global coordinate system. The scanner allows for truly reference independent 
scamung, yet it gives very precise description of the three-dimensional surface. 
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The caUbration. typically be peifonned once during manufacturing, which 
should be enough to last the life of the scanner. However the scanner can simply and 
quickly re-caUbrated if the need arises. 

A representative example of the calibration of the seamier wiU be better understood 

. from Figure 10 and tile followincT Hicrncci^r, ^ ■ -„ . .. 

w„.w... x iguie IV IS a uiustraaon ot an arbitrary, 

^q^ie ray R which is projected from the projection system of the seamier onto the 
smooth, planar calibration surface 138 of Figure 8 during the first part of the caHbration 
procedure.- The -ray-R^;^—is-captured -by -the electronic imaging-device 56, with the 
calibration plane positioned at two different distances from the seamier, Zl and Z2 The 
distance between the two locations is AZ. Distance Zl need not be known, however the 
separation distance AZ is known. The separation distance AZ wiU vary depending on the 
depth of focus of the imaging optics 108 in the scanner 14. 

Figure 10 illustrates a fundamental principle of the technique that is used for 
calibration of the seamier and generation of three-dimensional information of an object, 
which is considered to be an improvement over the calculations required by the method of 
Figure 9. Figure 10 illustrates that when the plane 136 is at a distance Zl, the ray R ^ 
impinges on the imaging device at tbe location 150. When the caUbration surface 136 is 
moved to position Z2, the ray R ^ ixnpinges on the detector at point 152 ITxe pixel 
coordinates for ray R ^ at both positions is stored in a calibration table. In actuaKty. the 
pixel coordinates for a large number of rays from the projection pattern are stored for Zl 
and Z2. These pixel coordinates, along with X and Y dimension measurements from a 
second part of the caUbration procedure, give all the information needed to create a 
caUbration table necessary to compute threenimiensional coordinates for an object that has 
been scanned with the projection pattern. 

• R n,n. corresponds to a single point in the projection pattern. Knowledge of 
where in the projection pattern ray R originated from is required. Hence, some pattern 
recogmtion and decoding of the detected pattern is needed to identify tixe specific portions 
of the pattern that are being imaged by the various portions of tiie CCD electronic imaging 
device. To miderstand the pattern recognition process, the reader is directed to Figures 1 1, 
12, 17, 18 and 19 and the following discussion. 

Pattern Repognitinn 
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Figure 1 1 is an illustration of a portion of a pattern that is projected from the scanner 
of Figure 3 onto an object (including the calibration surface 138). The projection pattern 
comprises an array of paraUel lines 156 separated from each other by red, green and yellow 
colored dots 158, it being understood that other types of projection patterns are possible. 
The sequence of the colored dots 158 vary along the length of the lines 156, and in the 
direction perpendicular to the lines. This technique is used such that using pattern 
recognition and decoding processes, described herein, every region in the projection pattern 
-canbe -decoded" fi-om-theptxel-data- from the imaging device. Figure-l-lillustrates that -Ray ■ 
can be taken to have originated at the intersection on one particular line and one 
particular colored dot, and this location can be determined precisely form the pattern 
recognition and decoding process. Similarly, the ray along the line L one row below ray R 
can also be identified. In the present example, the pattern is constructed such that there 
are N columns or lines, and.M rows of colored dots. Figure 12 illustrates showing that the 
various rays of Ught passing through the pattern of Figure 11 can be represented by an array 
of N X M points. 

This array of pomts representing the projection pattern of Figure 11 is imaged by an 
electronic imaging device or CCD 56 arranged in an array of row and columns, as shown in 
Figure 14. There are X columns of pixels in the X direction and Y rows of pixels in the Y 
direction. In the illustrated embodiment there are 1,028 pLxels in each direction. 

Figures 17, 18 and 19 illustrate the pattern recognition process for captured two- 
dimensional images. Figure 17 shows the signal level from one row of pixels in the 
electronic imaging device. The signal indicates that line L of the projection pattern is 
imaged at pixels 20-23 of a particular row of the pixels. Averaging of the pixel values over 
the four pixels allows the center poiat of the line relative to pixel 21 to be calculated, with 
sub-pixel resolution. Figure 18 shows how the line L and adjacent colored dots are imaged 
on the surface of the imaging device in this example. Note that the line is not necessarily 
centered over any column of pixels and hence the averaging must be performed to deteraune 
the center point in the line. A similar pattern recognition process is performed for the 
colored dots, as indicated in Figure 19. The center of each colored dot is located, as is the 
center points of the lines for every line and colored dot imaged by the imaging device. 

The pattern recognition process thus takes the output signals of the imaging device 
(in the form of a colored bittnap image) and returns a set of pixel locations for centers of 
lines and centers of particular colored dots. The next step in the process is correlating these 
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pixel locations for lines and colored dots to particular lines and colored dots in the 
projection pattern. 11ns process is referred to as decoding (process 82 in Figure 6), and is 
described in detail below. Decoding is not normally needed during the calibration 
procedure described in conjunction with Figure 8, since the Z calibration surface is planar 
and the arrangement of the projection pattern on the CCD 56 is preserved. Decoding is used 
however during use of the scanner to scan an object of unknown surface configuration. 

T)ecoaigg 

The decodmg process is the process of converting a set of pixel addresses for lines 
Hnaged by the imaging device, and a set of pixel addresses for particular colored dots 
imaged by the imaging device, to particular lines and colored dots in the projection pattern. 
Decoding is not absolutely required during caUbration (particularly where the Z caUbration 
• surface is a planar smface). It is used, however, during processing of images on an object 
havmg undercuts, shadow features, or other irregularities. It may be possible to decode 
only a portion of the received pattern, since ordering of lines in the projection pattern is 
preserved. For example, if hnes 13 and 16 are decoded, Unes 14 and 15 are also decoded 
smce theu: spatial,relationship relative to lines 13 and 16 are preserved. 

The imaging analysis process needs to know that a particular pixel is imaging a 
particular line or a particular colored dot. The projection pattern or screen 50 (Figure 2) 
vanes continuously in both directions, due to the unique and continually varymg sequence 
of the colored dots. The decoding process simply examines where the red, yellow and green 
dots are being imaged in the hnaging device, and compares these results with the known 
sequence of red, yellow and green dots in the projection pattern, and thereby locates or 
Identifies each ray with reference to the projection pattern. For example, the process knows 
that, for example, pixel 21 in row N of the imaging device is imaging the center of line 13, 
row 55, in the projection pattern. 

Referring again to the cahbration set-up of Figures 8 and 10, the seamier takes two 
unages of the Z-caUbration surface 138, one at distance Zl and the other at distance Z2. 
THe pixel addresses where each ray R in the projection pattern is imaged by the array is 
stored in a calibration table referred to herein as cahbration table # I, shown in Figure 24. 
At this point, we know how the imaging of the projection pattern varies as the caUbration 
surface is moved in theZ direction relative to some imaginary plane Zl in front of the 
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scanner. However, the X and Y relationship is not yet known. Therefore, the scanner must 
be calibrated in the X and Y direction using a pattern of known geometry. This is explamed 
in conjunction with Figures 8 and 13-16. 

Figure 13 is a illustration of the X-Y calibration surface 140 of Figure 8, showing 
5 the array of Q X P points (tiny apertures 143) in the caUbration surface 140 being organized 
into a coordinate system havmg an origin at the center of the surface 140, in the shape of a 
cross 145. The caUbration surface 140 is conceptualized as consisting of four quadrants I - 
_3^_^igure23-shows-oneT)ossibleTiumbering-convention-of-theT)oints-in-the-surf^ 

origin. In the iUustrated embodiment, the points of the X-Y caUbration surface 140 are 
10 actually tiny apertures spaced from each other a known distance (e.g., 1 mm). The apertures 
act as a pluraUty of point sources of Ught when the light source 142 positioned behind the 
surface 140 is activated. These points of Ught are inmged by the electronic imaging device 
56 during the second part of the caUbration step. By counting pixel signals (mdicating the 
imaging of a point source in the surface 140) over from the origin in the X and Y directions, 

15 it is possible to determine which point in the surfece 140 is being imaged by which pixel, 
again with subpixel resolution. Since we kaow the address of the pixels which illuminate 
the specific portions of the projection pattern, and we can know the distance from the origm 
of the surface 140 that this pixel is imaging, it is therefore possible to calibrate the pixels m 
the X and Y directions. A second caUbration table, shown in Figure 25, is used as an 

20 interim step to generate the distance values in the X and Y directions for the principal 

caUbration table # 1 in Figure 24. 

This process will be explained by example. Figure 15 is an illustration showing the 

interpolation of pixel addresses in X and Y directions for a given ray R ^ from a scanned 
object from two points of the X -Y calibration plane previously imaged by the electiwmc 
25 imaging device during caUbration. Figure 15 indicates that a given point in the projection 
pattern, ray R,,3, is imaged by some pixel that is located between the pixels that imaged 
points in surface 140 that are in quadrant IE, between 14 and 15 pomts to tiie left of the 
origin and between 14 and 15 points below the origin. This is suggested by Figure 21, 
which shows where ray R „ is imaged on the CCD chip. Figure 21 also indicates where tiie 
30 corresponding points on the X-Y caUbration surface 140 are imaged by the pixels in tiie 
electronic imaging device. As is shown in Figure 21, in tiie present example tiie pixel 
values are between 30 and 90 in the X and Y directions. 
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Figure 25 shows the X - Y calibration table # 2 that is used for generating distance 
entries in X and Y directions in the calibration table No. 1 shown in Figure 24. Figure 25 
illustrates that for each point in the X-Y caUbration surface 140. corresponding pixel 
addresses for pixels imaging those points are identified and stored in the table. This is done 
for aU points in the four quadrants of the X-Y calibration surface 140. These values are 
obtained when the X-Y calibration surface is positioned at both distances Zl and Z2, and an 
image is generated at both positions. The entries in the table are pixel addresses in X and Y 
directions, expressed-in-sub-pixel-resolutionr-Representative^ntries for Quadrant -I- are- 
shown, it being understood that entries are made for all the points in the X-Y caUbration 
surface. 

Now, if we know that ray R „ of the projection pattern from the Z-calibration 
procedure (using the caHbration surface 138) is being imaged at a particular location, we can 
use the calibration table # 2 of Figure 25 to compute an entry in mm for table # 1. Again, 
usmg the present example, assume ray R ,3 (corresponding to line 2, row 3 in the projection 
pattern) is-imaged by pixel having an address of 30.2 in the X direction and 36.2 in the Y 
direction, at Z = Zl. The distance in mm can be calculated from an mterpolation of the 
entries in cahbration table 2 of Figure.25. This is indicated for the entries in Ime 2. row 3. 
At the Z - Zl distance, this point is imaged at pixels 30.2 in the X direction and 36.2 in the 
Y direction, which corresponds to a distance of-14.6 mm in the X direction from the origin 
and -14.4 mm in the Y direction. Shnilarly. at Z = Z2, this point is imaged by pixels 43.0 
m the X direction ^d 31 in the Y dfrection. which corresponds to a distance of -14.8 mm in 
the X direction and -15.8 mm in the Y direction. Tins information is entered into table # 1, 
as shown in Figure 26. 

This interpolation takes advantage of a linear relationship that exists between pixel 
address and distance for objects at an unknown distance Z ' from the scanner. This can be 
best appreciated from Figure 22. Since we know that ray R _ is imaged at one point 160 
m the imaging device at Z = Zl, and that it is imaged at another point 162 at distance Z = 
Z2, the ray must fall along the dotted line 164 where Z ' is between Zl and Z2. Similarly, if 
Z • > Z2 it lies along the line indicated at 166. If Z ' < Zl, it Ues along line 168. TMs 
bnear relationship between distance and pixel address is the key to obtaining Z mfoxmation 
as. to ray R and X and Y distance in mm. Furthemiore, smce AZ is known exactly 
duimg the caUbration (e.g.. 7 mm in the iUusfrated embodiment), and a linear relationship 
exists between pixel address and distance, the location of exactly where ray R ^ Hes along 
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the line 164 166 or 168 teUs us with precision the position of the unknown object in the Z 
direction, relative to virtual plane Zl. This location of where ray R ^ is being imaged by 
the electronic imaging device " is arrived at by the pattern recognition and decoding 

processes described herein. 
5 Rrfening again to the example of Figure 16 and 17, we know ray R „ is toa8«i ^ 

some region of the CCD, and this is stored in table 1 (Figure 25). Frona caUbration table 2 

(figure 06) we know the X and Y coordinates of neighboring points in the X-Y calibraBon 
_p;gT5T55T5^gE,rnr-R:^BriBteW^ 

calculate (he distance ia nm. fom flie origin since the points in the X-Y calibration gnd are 
10 separated fiom each other a known distance. Tins is done &r all N X M portions of te 

projection pattern. j • 1 

For example, caHbration table 1 of Figure 24 tells us that ray R „ is imaged at pixe^ 
addresses X = 30.3 and Y = 36.2 for Z = Zl, and at pixel address X = 43 and Y - 31 at Z 
Z2 We then look to the table 2 entries (Figure 25) to find the closest X and Y pomts m tiie 
15 X-Y caHbration grid by looking at the pLxel addresses in table 2. Tins is shown m Figure 
16. An interpolation of the pixel addresses in table 2 to the known pixel address firom table 
1 results in an X. Y address in mm. In the present example, X in mm = -14.6, Y m mm 
14 4 The results are now added to table 1, see Figure 26. The same is done for the distance 
Z = Z2 The same is of course peifonned for all the N X M rays in the projection pattern, 
20 resulting in a completely filled caHbration table 1. Table # 1 in Figure 26 only shows the 
entries forrayR, 3, buttheprocessisdoneforaUraysfor Z = ZlandZ-Z2. 

Figure 20 illustrates the points of the X-Y caHbration plane firom lie calibration 
station of Figure 7 at two positions Zl ands Z2, relative to the optical elements of &e 
electronic imaging device. It will be apparent that when the Z = Zl, some points m the X- 
75 Y caHbration plane will not be imaged, but will be when Z = Z2. Tlxese points, mdicated by 
AQ in the Q direction, exist also in the P direction. The calibration table 2 takes these mto 
account. Some points in all four quadrants may not be imaged at Z = Zl , but will be maaged 
at Z = Z2 during X-Y calibration. Points indicated at 170 are imaged at both values of Z. 

From the above, in one aspect of the present invention, a machine-readable memory 
30 is provided for a seamier used to calculate three dimensional information of an object 
scanned by the seamier. TTie memory may be in the scamiing miit itself; in a separate work 
station for the seamier, or in any computing device such as a remote computer that processes 
acquired image data and generates a three-dimensional model of the object. The memory 
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comprises an array of data storage locations containing a calibration relationship for the 
scanner, such as a table. The calibration relationship identifies pixel coordinates for 
numerous portions of a pattern projected onto a caUbration surface located at t^vo different 
distances from the scanner, and distance information in X and Y directions for the portions 
of the pattern for the two different distances. The calibration entries stored in memon- 
aUow the scanning system to compute three-dimensional coordinates for points on an object 
reflecting the projection pattern onto the electronic imaging device. 

-Now^hatthe-scamieriias-been-completelrca3ibratedri1^-^^^^ 
some unknovvn distance and having some unlmown surface configuration. The derivation of 
X, Y and Z coordinates for the surface will be explained in the next section. 

. Derivation of 3-D Point jn^^pp r.f^ «^ 

• With reference to Figure .6 and 26, we now explain the derivation of spatial 
coordmates in three, dimensions for a single captured image. With the entries in Table 1 
completely filled out during caUbration, the seamier is now able to derive X Y and Z 
•coordmates for any object at an unknown distance. The seamier has the most accuracy 
when the distance is between the values Zl and Z2, such that the captured images are in 
focus, but distances outside of this range may still be able to be imaged and decoded. 

First, the electronic imaging device 56 captures an image and the image is subject to 
the pattern recognition and decoding, steps 80 and 82 in Figure 6, described in detail above. 
This process results in a set of pixel coordinates stored in memory for aU the lines and 
colored dots of the pattern that are projected onto the object and imaged by the pixels of the 
CCD. Comparison of the pixel addresses of the captured image with the entries in Table 1 
m Figure 26 (when completely fiUed out during caUbration) yields the coordinates of every 
pomt m the captured unaged, in three dimensions. 

The process is as follows: 

Fkst, compute the Z value of every portion in the projection pattern found in the 
captured miage using table 1, given the known Une and row number of the portion of the 
pattern, and the associated pixel nmnber. THe unknown distance, Z for any point, 
measured from the virtual plane Zl is as follows 



Z ' (in mm) = 
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AZx measured pixel # - pixel # for line and row of p attern at Zl 

. . ■ pixel #■ for line and row of pattern at Z2 - pixel # for line and row of pattern 

atZl,. 

where A Z is.the distance from Zl to Z2 in the calibration set up described above. 
. . ■ Using ray as an example, if this ray is imaged at pixel # 35 in the X direction, 

■I ■ ' * * *• • . * 

from table 1 the calculation is as follows 

Z'(in"mm) = 7.0 mm x 35 - 30.2 w here A Z is 7 mm. 

43-30.2 

= AZx a Here, 0 = 0.375, a linear scaling factor. 

Therefore Z ' = 0.375 X 7 mm or 2.625 mm. The point on the object reflecting 
ray R 2,3 is 2.625 mm from the virtual plane Zl. The Z value for all other points in the 
object are also measured relative to a virtual plane Zl . 

Now, Table 1 (Figure 26) is referred to determine the mm values in the X and Y 
direction for this point in space and another interpolation is performed to the mm entries in 
table 1. The calculation is as follows: 

Since we know we have line 2, row 3 in the pattern (from the pattern recognition and 
decoding process), we need to interpolate the rmn entries in table 1 . 

X value is between- 14.6 and - 14.8 AX = 0. 2 mm 

Y value is between - 14.4 and -15.8 AY =1.4 mm 

The true value of X = X ^tzi - (ox AX), similarly the true value of Y -Y uzi- (-^ 

X AY). 

Therefore: 

The true value of X for ray R ,4 = - 14.6 - (0. 375 X 0.2) = -14.675 mm 
The true value of Y for ray R ,.3 = - 14.4 - (0.375 X 1.4) = - 14.925 mm 

Summarizing, the X, Y and Z coordinates for the point in the object reflecting ray R ,3 is 

X = - 14.675 mm 

Y =- 14.925 miii 

Z =-2.625 mm 

These points are stored in memory 42 of the scanning work station 16, step 86 of 
Figure 6. This process is performed for every ray R in the projection pattern that is 
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recorded in the captured image. The result is a set of three-dimensional coordinates in 
space, referred to herein as a "frame", for each captured image. This set of points can be 
calculated in a small fraction of a second with a general-purpose computer. 

The pattern recognition, decoding and 3-C coordinate, calculation process will now 
be explained with reference to two-dimensional bitmap images of teeth. The process 
described below is the same for any object being scanned. 

"Part "ST. GeneratTo n of Djgjfal Impfessinfi" 

A complete three-dimensional model of the patient's dentition can be generated from 
the scanning system of the present invention. The process requires an operator moving the 
scanner 14 (Figure 3) over the dentition, preferably in-vh'o, and generating a se: .:-5 of 
frames of images. The frames are obtained at a rate of at least one frame per second as the 
scanner is moved over the teeth. The scanning of an entire jaw may require three separate 
scanning operations or "segments" due to maneuverabihty constraints and a break in the 
c^turing of images. While the scanning is occurring, the four steps of Figure 6 are 
performed for the stream of c^tured images. The end result is the storage of a set of frames 
in the main memory of the scanning work station 16. The frames can be registered to each 
other using one or more of a using a variety of frame registration techniques described in 
detail below. Once all the frames have been registered to each other, a complete three- 
dimensional virtual model of the patient's dentition is displayed for the orthodontist. This 
computer model provides a base of information for diagnosis and planning treatment. An 
introduction to the treatment planning aspects of the overall orthodontic care system is set 
forth in Part 4. 

Figure 27 is an illusfration of a two-dimensional bitmap image of a tooth and 
associated anatomical structures c^tured by the electronic imaging device 56 of the scanner 
of Figures 1, 2, 3 and 4, prior to any signal processing in the scanning work station 16. An 
inspection of Figure 27 indicates that the image includes various lines and colored dots of 
the projection pattern, as it is reflected off of the tooth and associated anatomical structures. 
The location of where these lines and colored dots are imaged on the imaging device 56 
contains information as to the three-dimensional shape of the tooth and associated 
anatomical structure. 

Referring back to Figure 6, the first step is a pattern recognition process on the 
c^tured image. Figure 28 is an illustration of the image of Figure 27 after pattern 
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recognition and filtering operations have been perfonned. The filtering operation basically 
returns a zero pixel value for those pixels where the pattern recognition process cannot 
detect the lines and the colored dots, for example due to the surfece being out of focus (i.e., 
out of range of the scanner optics). 

After the decoding operation is done on the image of Figure 28, the result is a set of 
pixel locations where the decoded rays R of the projection pattern were imaged in the 
imaging device 56. Step 84 is perfonned for all such rays, using cahbration table # 1 of 
TigurelZ B stored m memory f or the scanner. IhFf^rstqp^^'on^iguje-eTira^t'of 
dimensional coordinates for all the points in the image, a point cloud comprising a "frame. 
Figure 29! is/an illustration of a single "frame" of data, that is, a three-dimensional point 
cloud of a scanned object which has been calculated from a single two dimensional image 
by the pattern recognition, decoding, and 3-D calculations described herein. 

Figure: 30 is an illustration of the points of the cloud of Figure 29, in which three 
adjacent points' of'the cloud are joined together to form triangle surfaces. The usage of the 
15 triangle surfaces in a registration process is described below. Figure 3 1 is another view of 
the three-dimensional surface formed from the triangle surfaces shown in Figure 30. Figure 
32 is a view of the surface of Figure 31, smoothed by a smoothing algorithm to give a 
smootherrepresentatip^ of the surface of the object. Commercially available off-the-shelf 
software exists for taking a set of three dimensional coordinates and displaying them on a 
20 computer monitor, and such software is used to display the three dimensional surfaces (if 
desired by the user). 

Figure 33 is another example of a bitmap image obtained by the electronic imaging 
device of the scanner. Figure 34 is a plan view of the three-dimensional surfece obtained 
from the two-dimensional bitmap unage of Figure 33, after the st^s of Figure 6 have been 
25 performed. 

Figure 35 is a perspective view of the three-dimensional surface shown in Figure 34. The 
software programs enable the surface to be rotated in any degree of freedom, allowmg for 
complete inspection and visual analysis of the surface. 

Since the scanner and scanned object move relative to each other during capture of 
30 the scanned images, the three dimensional coordinates for a large number of fiames will not 
agree with each other. In other words, the X, Y and Z coordinates for a given point on the 
object will change from firame to frame since the pouit was imaged from a different spatial 
orientation for each image. Hence, the frames have to be registered to each other to 
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generate a complete overaU digital n.odel of the object. The present invention provides for 
venous registration procedures to be perfonned on the frames, to find a best -fit solution for 
coordmates of the object in one fr^e vis-a-vis coordinates of the object in other frames. 
These registration procedures are described in the following section. 

Figure 36 IS a flow chart llTnQtra+i'Tnrr i-U^ ^^^.^^ ^ n . , 

c^i — -^^vxx^g uxe :.tcps penoimea to generate a complete 
three-dimensional model of the dentition of a patient from a series of scans of the upper and 
^er jaws.^e steps include an initial step 190 of determining an enHy point mto a 
registration P'^ocess;^ame-fo-ffimere^s1^^ 

another, a segment registration process 194 to register segments (i.e., portions of a scan 
where the scan was interrupted for some reason) to one another, and finally a cumulative 
registration procedure 196 in which the frames are registered to all other frames to obtain a 
shghtly more accurate model of the object than that obtained by frame to frame registration. 
It will be understood that depending on the apphcation, the step 194 may not be required, a 
frame to frame regisfration may only be needed, or the user may only desire a cumulative 
registration and steps 192 or 194 are not performed at aU. 

The result of registration is a three-dimensional model contaming all the pomts from 
the frames captured by the scanner. An example of such a model is shown in Figure 37A, 
where surface 199 indicates the smn total of points of all the frames in three-dimensional 
space. Figure STB illustrates one smaU section of the surface, showing for example the 
pomts m three-dmiensional space fix>m tiiree frames. 

A preferred registration technique involves registering a set of points (three- 
dmiensional coordinates) comprising a frame to a surface formed by a previous frame (or 
groiq, of frames), rather than registration of one set of points to another set of points. TTiis 
IS due to a relatively coarse projection pattern used in the illustrated embodiment; the points 
can be low in density as compared to the curvature of the object. Figure 37C shows one 
frame from a scan of a tooth, with the points in the frame comiected by lines to form a set of 
tnangle surfaces. The coarseness of the projection pattern (and widely spaced points in the 
pomt cloud) is compensated by the fact that a given portion of the surface is captured by 
overlappmg frames, and the registration occurs from the points in one frame to the surface 
in the previous frame or a surface defined by more than one previously registered frames. 
The registration process described herein ultimately pemiits a fine resolution in the three- 
dmiensional model. Hiis is indicated by Figure 37D, showing all the frames for the tooth 
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scan registered to each other to create a very fine and high resolution*virtual model of the 
tooth. 

A. Entry point into Registration Tstep T 90. Figure 36) 
5 Registration processes require a starting point for fitting one tame, fi-ame i to 

another frame, frame i + 1. The starting poiat, in the illustrated embodiment, is rough 
calculation of the offset between overlapping points in the frames in X, Y and Z directions. 
Whereas pr i6f"art systems Eve good pre-lmowledge of tKe^ptiaTel'atioh^ due 'to the 
known physical arrangement of the scaimer and tite object, the present system does not. 
1 0 The starting point is tiie initial assumption of spatial relationship between one frame and the 
previous frame (and one frame and a set of previous frames). 

The method of calculation of the offset in X and Y directions is illustrated m Figure 
38A-C. Figures 38A-38C are an illustration of a two-dimensional cross-correlation 
procedure in the X and Y directions. The procedure, along with the procedure in Figures 
15 39A and 39B, is used to find an initial entry point into a registration algorithm betweai 
successive frames of data. 

It can be seen from Figures 38A and 38B that frame i + 1 is moved in the X and Y 
dkections from frame i. To obtain the amount of movement, AX and AY, the images are 
projected to both X and Y axes by adding all pixel values of every row, acquiring a one 
20 dimensional Y-vector, .aud adding all the pixel values of every column, acquiring a one- 
dimensional XTVgctpr. This is performed for both frames, resulting in X ^am^ i ^ fi™» ' + ^ 
and Yfc^jand Y ^^i^i The vectors are smoothed to suppress the influence of the projected 
pattern. 

To compute AX, the absolute value of the difference between each value of the X- 
25 vector of frame i (frame i-X ^diheX vector of frame i + 1 (frame i + 1 - X i + 1 ) 
is calculated with a varying position shift within a range of -xa < k < +xe. The sum of 
these values represents the resemblance of X ft,,. ^ and X i + 1 shifted by a certain amount 
k. The minimum value of k is determined. This result gives the shift or movement m the 
AX direction. 

30 The same process is also performed the Y direction. As can be seen in Figure 38C, 

if frame i is moved by an amount AX and AY, the overiapping points in both frames will 
have the same values and the sum of the difference in pixel value will be approximately 
zero. 
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Fig>«e 39A and 39B are an iUusttation of a one-dimensional correlation procedure in 
the Z dnecaon for two succeasive ftamea, in the present example frame 1 and ftame 2. 
Lme 200 represents a three-dimensional surfeoe of the second ftame. fonned by connecting 
three adjacent points in the point cloud. The set of points 202 represents the points of the 
pomt cloud representing frame 1. To compute the Z offset, A Z. the sum is taken of all the 
Z values of Frame 2. and the resuh divided by the number of points, resulting in an average 
Z value Of Frame 2. Tbc same is done for Frame 1. The diffirence between Are average Z 
v-aueoTTrame ^rmnS^S,,m^rvSl^tTsm^-T^TsrTo^,,Zr^.-37B 
tllusnates the result ate ..Nation of Frame 1 by Are amount AX. AY and AZ. The result 
.s that thepoirrts of frame 1 are tpite close to the triangle surfices of Frame 2. The values 

of AX. AY and AZ are stored for Frame 1 and used a. an entry point to a registration 
procedure. 

B. Frame to Framft RegistraHnn 

Frame to frame registration is a process for registering one ftame with another 
frame, that is. finding a best-flt in terms of translation and rotation make overiapping points 
m the frames agree with each other. If the flames are generated in sequential order, frame 
to frame registration refers to registration of the second fl^e to the firs, frame, the third 
ftame to the second feme, flom the fourth ftame to the ttnrd Same. etc. Frame to ftame 

registration can be performed very quicklv It can he t,*^™^^ ■ , ^ t. 

J- Hiuv^aay. II can De performed m a manner such that the 

operator of the scamnng system sees the results of ftame to ftame registration on the 
momtor of the scanning wort station- v>Me flrey are stiu scanning the patient mat they 
see ,s an accurate Ihree^dnnensional representation of Ure dentition on the momtor, for those 
pomons of the dentition that have been scanned thus far. As additional frames are obtained, 
they are registered to the previous frame and added to the computer model. When scamnng 
IS complete, the computer model can be rotated around on the monitor and inspected to see 
that all relevant portions of the teeth ind anitomical stmctires have been scamied. The user 

thus gets unmediate feedback on the results of „ ■ ■ « 

uu me results ot the scamnng usmg frame to frame 

registration. 

Figures 40A-40D is a flow chart of a ftame to frame registration process for a set of 
frames, each frame consisting of a three-dhnensional point cloud of a scanned object. Each 
frame rs tjprcally generated from a different spatial oHentation of the scanner relative to the 
object due to movement of the scanner during image capmre. hence the frames overlap to at 
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least some extent. The registration process is used to find a best fit between the frames 
relative to.- each other, and thereby provide a complete three-dimensional virtual model of 
the surface of the object firom all of the frames. The end result of the frame to frame 
registration is a substantially exact three dimensional model of the scanned object. This 
5 object is represented by a large set of point coordinates in computer memory. The result is 
also represented as a set of transformation matrices providing information as to how each 
frame of points should be translated and rotated in three dimensions in order to fit to the 
previous frame. 

The frame to frame registration process is also an iterative process. At the end of 
10 each iteration, a comparison is made as to hov^ "close" the two frames are to each other in 
three dimensions. If they are not close enough (with "closeness" determined in absolute 
terms by a quality index, say in microns), another iteration is done, using the results of the 
first iteration. The frame to frame process may continue for tens or even himdreds of 
iterations, depending on how fine or precise the user wants the registration to be. The 
15 process stops when a best fit between two frames has been reached or a maximum number 
of iterations has occurred. 

Referring now to Figure 40A-40D iq conjunction with Figures 39 and 41-43, an 
initial step 209 is performed as an entry point into the registration procedure. In this step, 
the processes of Figure 39A and described previously is performed to make a Z-coordinate 
20 transformation of frame i-1 and fi^e i. Basically, this transformation is performed by 
summiag the Z coordmate values of both frames individually, finding a median Z value for 
each framcj findmg the difference or A Z value from these median values, and shifting one 
frame (firaine i) by this amount to bring the Z coordinates of all the pomts in that frame 
closer to the points of frame i-L The rationale of step 209 is as follows. The Z axis 
25 measurements of a surface represent a degree of freedom of the scanner movement. 
. Scanner movements in the Z dhection do not provide significant new information on the 
object being scaimed. However, because of the focus optics of the electronic imaging 
device (ICS in Figure 7), the visible surface of the object becomes slightly smaller or larger 
depending on how much the scanner is moved in the Z direction relative to tiie surface, 
30 while tiie center of the surface (or, loosely speaking, tiie "center of gravity") of the visible 
area basically remains nearly the same with Z direction movement. The Z coordinate 
transformation step 209 ehminates this effect by normalizing tiie Z surface coordinates. 
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This process also makes possible the exclusion criteria of step 2 described below, bv which 
non-overiappingpoints and stray datapoints are excluded from the registration process. 

The registration procedure itself starts with step 1, 209 in Figure 40A. At this step, 
nmmnum distance vectors 248 (Nl. N2, . . . ) are calculated from every point in Frame i to 
the surfece of frame i-1. The surface of a frame can be obtained easily by connecting 
neighborhood points to together with a triangle or other polygon. Ite minimum distance 
jecto^r eachpoint in Frame i, is defined as the distance vector having a magnitude which 
IS the mmimum of-fETIfolISwinrWeeTvec^^ 

mtersectmg a triangle surface in frame i-l nonnal to the triangle surface; 2) the shortest 
vector from the point orthogonal to the edge of a friangle surface in frame i-l, and 3) the 
shortest vector from the pomt to the nearest point in frame i -1. Most often, but not 
always, this wiU be a nonnal vector, type 1) above. In the example of Figure 41, minimum 
distance vectors 248 are computed from the points 250 in frame i to the triangle surfeces 
252 of fr:ame i - 1, with the vectors 248 nonnal to the surfaces 252. 

At step 2 (212 in Figure 40A), three exclusion criteria are apphed to the minimum 
distance vectors of step 1, in order to ehminate non-overlapping datapoints between the two 
frames and to ehminate stray data points. First, all minimmn distance vectors that relate to 
a boundary element (edge or point) in frame 1 are excluded. Second, aU remaining 
mmimmn distance vectors with an amomit exceeding a certain predefiued value R, likely 
mdicatmg stray data pomts, are excluded. Thirdly, only triangle surfaces are taken into 
consideration which form the outside surface with respect to the seamier viewmg direction. 
Every surface has by definition two sides. We regard the "outside" surface as the surface of 
the object that is oriented towards the scanner. 

At step 3 (214) the vector smn of all the minimum distance vectors N ^ . . . N „ is 
computed. This is shown in Figure -42, with vector 254 representing the vector sum. 

At step 4 (215), the median minimal distance vector (t) is computed by multiplying 
the vector sum 254 by the scalar 1/N. The median minimal distance vector basically 
constitutes a measure of how frame i should be translated m X Y and Z directions in order 
to better fit to frame i -1. Now, the registration process needs to compute a rotation factor, 
which IS explained by steps 5-8. to indicate how frame i needs to be rotated in order to better 
fitfi-amei-1. 

At step 5 (216), the X, Y and Z components of the median minimal distance vector 
is subtracted from every point in frame i. This is performed by making a copy of the frame i 

60 



WO01/8(.761 PCT/USOl/11969 

coordinates and operating on the copy as an interim step in the procedxure, the underlying 
data from frame i is unchanged. At the same step the "center of mass" of the points of 
frame i which are not excluded by step 2 is calculated. The "center of mass" is defined as 
the vector suin of position vectors of all mentions points scaled by the inverse of the number 
of points. 

At step 6 (21 S) a calculation is made of the cross product of two vectors for every 
point in frame i. With reference to Figure 43, the two vectors are as follows: 1) a position 
vector'Vi ^^ding 'from~me 6rigiii~2"53 of "tfie~global coorainaie~^steml:o"the points in 
frame i, subtracted by the vector of the center of mass of the remaining points of frame i as 
calculated in step 5, and 2) the identified minimum distance vector Ni for that point. 

At step 7 (220), a calculation is made of the vector sum of the cross vectors 

calculated in step 6, that is the net cross vector ^^(v^. x rij) for all i points in the frame i, 

J 

where x is the cross product operator. 

At step 8 (222), the vector sum of step 7 is weighted against the inverse of the sum 
of all squares of the position vectors (Vi) of the points in frame i, to arrive at a rotation 
vector U. U is interpreted as follows: The direction of U gives us the rotation axis and the 
magnitude of U is the angle or amount of rotation. In particular, if we consider Vi to be the 
position vectors from the origin of the coordinate system to the vertex of every point, and Ni 
being the Tninimal distance vectors defined above, then the weighting is as follows : 



J 



The reasoning behind this weighting is as follows. If you imagine the distance 
vectors as the realization of linear spring elements, the vector sum of the cross products 
represents the aggregate moment, or rotational discrepancy, generated between both frames, 
la the case of small deviations between the position of firame i and its final position, it can 
be assumed that the rotational moment also determined the direction of the necessary 
adjustment. The scaling with the help of the inverse of the sum of tiie squares of the 
position vectors considers the global extension of frame i. That is, the larger the distances 
of the points from the center, the larger is the ratio of rotational moment and angle between 
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the present position and the target position. In a global sense, the mentioned factor (inverse 
of the sum of squares of position vectors) describes this ratio. 

The derivation of the proper scahng factor is by no means an exact calculation. It 
has, however, turned out that using this factor in all empirical cases, the iteration of defining 
overlapping areas and execution of transformations converges. 

At step 9, the result of step 8 is scaled with an empirical "acceleration fector" f The 
factor f serves to possibly accelerate this convergence. A value of f of greater than 1 is 
appropria^f^elativel5rlafp-rmaflonal-^^lac^^ to-be" 
determined empirically. 

At step 10 (226), the result of step 9 is interpreted as aa axis of rotation, the 
magnitude of which indicates the amount by which frame i has to be rotated in order to 
make the local overlapping areas of both frames he within each other. The magnitude of 
the rotation vector is mteipreted as the angle around wMch frame i has to be rotated. 

A rotation transformation matrix [T] (R) is calculated for frame i. This fonnula 
shows how to convert the rotation vector resulting from step 9, where p is the original 
length of the net cross vector which equals the angle of rotation that is required to fit the 

overlapping areas of frame i to fi:ame i-1 and u is the unit vector of U m = — with 

' \tt\ 



\U 

conq)onents ti^,Uy,u^. 



[r](J2) = 



(l-cos^z^; + cos;7 (X-cosJJ)u^u^-u,synj3 (l-cos jS^u^.+u^sm/S" 
(1-cos ^u^ti^+u^ sin^ (l-cos>7)«J +COS jff (1-cos j^u^u, -u, sin A 
^(l - cos ^u,u^-u^ sin ^ il-cos j3yti^Uy+u^^j3 (l - cos jSyu; +cx>&^ ^ 



To obtain a unique transfonnation operator for calculating the translation and the rotation 
transformation m a closed manner a 4x4 matrix representation is used. The relation bet^veen 
the 4x4 representation and the three dimensional rotation represented by 3x3 Matrix [T](R) 
is as follows 



25 [rj(i2) = 



0 1 



and between the 4x4 representation and the three dimensional translation, represented by 
vector (t) 
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[r4](0 = 



^10 0 O 

0 I 0 

0 0 1 

^0 0 0 1 y 



To apply, this 4x4 matrices on the three dimensional vectors the following convention is 
made: A three dimensional vector is transformed into the 4 dimensional vector space by 
identifying the. first three components of the 4 vector with the components of tiie three 
5 dknensional vector but.thlfourth component is always unique. {x,y,zy ix,y,z,iy 

At step 11 (228 in Figure 40 C), a transformation matrix for firame i, [T4](i), is 
calculated by multiplying the rotation matrix [T41 (R) (firom right) by the translation matrix 
[T4](t).from step 4 [rj(0 = [T.m • 

Altematively, the point cloud of frame i can be separately and independently 
1 0 operated on by the rotation matrix and the translation vector. 

At step 12 (230), a calculation is made of the square root of the sum of the squares of 
the minimum distance vectors calculated in step 1 (210) of Figure 40A, which indicates the 
closeness factor quality of the registration in this iteration, value MA below. At step 12 
(232) the closeness factor MA is compared to a quaUty index or threshold indicative of a 
15 successful registration (e.g., 50 microns). If the value MA is greater than the quality index, 
the registration process is repeated another iteration. All of the points in the ith frame are 
updated by operation of the transformation matrix [T4] as indicated at step 234. The 
process goes back to step 1 as indicated at 236, and another iteration is performed of steps 
1-13. 

20 If the closeness factor MA is less than the quality index, the registration process 

proceeds for the next frame. As shown in Figure 40D, at step 14 tiie process retrieves tiie 
next frame (frame i + 1) and frame i after application of the transformation matrix [T4] i. 
At step 15 (240), the iterative process of steps 1-14 is repeated for frame i and frame i + 1. 
This process continues until all the N frames have been registered, as indicated at step 16 
25 (242). At the end of the process, the result is a set of points comprising tiie tiiree- 
dimensional model, and a set of transformation matrices [T4] ^ to [T4] n for each of tiie 
frames (otiier tiian .tiie first frame, firame 1), which are used to generate tiie tiiree 
dimensional model. One frame, such as frame 1. tiie first one generated, is cbosen as tiie 
starting frame for the registiation and there is no transformation matiix for tiiat firame. 
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Figure 45 illustrates th« a nmge variable R can be used to filter or limit the nmuber 
of pomts m a fi-ame that are used when calculating net nonnal vector., triangle surface, 
cross vectors, etc. Tie purpose of flie range variable R is to screen out stray data points fir 
from the triangle suriace of the previous frame. The stray data has the effect of skewing the 
entires of the transformation matrix and increasing the number of iterations untB the quality 
mdex rs met R can be fer example 1 mm. or much less such as 1/50 mm. 

Figure 46 illustrates two ways in which the frame to frame registration process may 
tenmnate. As shown by line 260, tte process rhay terminate when the closeness factor goes 
below the threshold, here 30 microns. Alternatively, the closeness factor may level otf at 
some value above the quaUty index, despite further iterations of steps 1-13 of Figure 40. 
Thrs ts mdicated by line 262. When the closeness fictor MA does not hnprove by at least 
some amount per iteration (say 1 percent) after 10 iterations, the process is deemed 
complete. Here, the designation x indicates the iteration in which the amount of 
mrprovement in the quaUty index from the previous iteration to that iteration was first less 
than a threshold, e.g., 1 percent 



^- Cuillhlnriv»RB<ri.aT^t;n.. of KnHr« T^,„ 
As noted above, cmnulative registration is an alternative or improvement to a frame 
to frame registration. The difference between the two is that frame to frame registration 
only regrsters ™,e frame to one other frame, whereas cumulative registration is a registration 
of a frame to more than one other frame, such as where one frame is registered to aU 
previously registered flames. There are numerous types of cumulative registrations that can 
be performed, and a few exan^les will be given here. An advantage of cumulative 
regtstrahon is more accuracy in the resulting three-dhnensional model. He disadvantage is 
that cumulative registration can be significantly more computationally mtei^ive, and 
flterefore require more time to be perfomred using currenUy avaiW,le low cost 
microprocessors or computers. 

Figures 47A and 47B are a simpUfied illustration of one possible cmnulative 
Kgrstratron procedure. Figure 47 A shows triangle surfaces of frame 2 and points of frame 
1. The pomts and surfeces represented in Figure 47A actuaHy are in three dimensions. 
After regtstiration of fiame 2 to frame 1, we are ready for frame 3. Unlike frame to frame 
regrstratton, frame 3 is registered ,» both frame 1 and fiame 2. In other words, triangle 
surfeces are formed on the points of frame 3 and minimal distance vectors are calculated fer 
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each of the points in frame 1 and frame 2 (after registration to frame 1), and the other steps 
of Figure 40 are performed. As each new frame is retrieved, it is registered to the total sum 
of points from all the previous frames, after the previous registration. 

Figure 48A-48C are a flow diagram of one possible cumulative registration process. 
5 At step A (270), the points of the first frame are retrieved froin memory. 

At step B , the points of the second frame are retrieved from memory. 
At step C, the transformation matrix [T] ^ is retrieved for frame 2. This process 
assumes that the transformation matrix for each frame has already been generated, such as 
after a frame to frame registration process has been performed. 
10 . At step D, the transformation matrix [T] 2 is applied to frame 2. 

At step E, a registration is performed of the points of frame 1 to the points of frame 
2, after the transformation matrix [T] 2 has been apphed to frame 2. Basically, the steps 1-1 1 
of Figure 40A-40B are performed. 

At step F, a check is made as to whether the quahty of the registration is less than a 
15 threshold. If not, the registration is performed again (with the points of frame 2 updated by 
a new transformation matrix). Steps 278 and 280 are performed over and over again until 
either the quality threshold is met or a maximum number of iterations has occxirred. 

If the index is met or the maximum number of iterations has been reached, the 
process proceeds to step G (282). The new transformation matrix for frame 2, designated 
20 [T] 2 ' is obtained and stored. 

At step H, the new transformation matrix [T] . ' is apphed to the points of frame 2. 
At step I, the new transformed points of frame 2 are added to a "global container''. 
The global container is merely memory locations containing the points from frame 1 and the 
points of frame 2 as transformed. 
25 At step J, frame 3 and its transformation matrix [T] 3 is obtained from memory. 

At step the transformation matrix [T] 3 is applied to Ihe points of frame 3. 
At step L, a registration is performed of frame 3, as transformed, to all the points in 
the global container. Steps 1-11 of Figure 40 are performed. 

At step M, a check is made to see if the quality index is below the threshold. If not, 
30 another iteration of the registration process is performed. This repeats until the quality mdex 
is below the threshold or a maximum number of iterations is reached. 

If the threshold is met (or the maximum number of iterations is reached), the process 
proceeds to step N. The new transformation matrix [T] 3 ' is obtamed and stored in memory. 
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At step O, this new transfoimation matrix is appUed to the points of frame 3. 
At step P, the points in feme 3 after the transfoimation operation is perfomied are 
added to the global container. 



frames. 



At step Q, the process of steps A-P of Figure 48 are perfomied for the rest of the N 

At step R, all the transfonnation matrices [T] V • • . [T] ^ ' are stored on the hard disk 
of the back office server. These transfonnatiou matrices are used whenever the finished 
global container (complete three-dimensional model) needs to be generated again at a later 
date (or on another workstation). The model is generated by simply applying [T] • • - [T] n 
• to the raw frame data comprising frames 2 . . . N. 

At step S, the global container is displayed to the user. This can be on the monitor 
of the back office server 28 or on the monitor 20 of the scanning station 16 (Figure 1), 
Smce the global container is a digital representation of the object, it can be transported 
across a computer network and displayed and shared by another terminal. For example, 
where the back office server has a comxection to the Internet, the model can be transported 
over the Internet to the precision appliance service center 26 and displayed there. It can 
^so, for examples be shared among various orthodontic, periodontal or dental speciaKsts so 
that feey can collectively study the patient remotely and cooperatively plan care. 

Figure 49 is an illustration of a set of frames, illustrating a variation from the 
cumulative regisfration set forth in Figure 48, In Figure 49, a different order of fi^e 
regislratzon is performed from that of frame to frame registration. In particular, frame to 
frame registration and cumulative registration of Figure 48 are performed in the order in 
which the frames are obtained. This need not be the case. H fact, a more accurate 
registration may be obtained by registration of frames in an orxier based on 
'•neighborliness". that is, based on the concept that aU frames imaging a particular portion of 
an object should be registered together (or in sequence such they are registered one after the 
other), nie order of regisfration is indicated by the left hand column of numbers. The 
nght hand side of Figure 49 merely illustrates that each frame consists of X, Y and Z 
coordmates for a set of points. The frames need not and usuaUy will not have the same 
number of points. 

In Figure 49, the regisfration order is based on the location on the surface of the 
object for a given frame relative to the location on the surface for other frames. Figure 50 
IS a smiplified illusfration of a set of frames, showing the order in which the frames were 
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obtained, with the neighborUness of the frames relative to other frames being the basis for 
the regisfration order shown in Figure 49. Frame 7 (F7) is the chosen starting frame. So, 
ihe frames surrounding frame F7 are registered next This is shown by the order of 
regisfration being F7, F3, F4, FU, F12, F6, ... as indicated. 
5 Figure 51 is another illustration of a set of frames, with registration of frames 

performed in accordance with the method of Figure 49. The order of registration is indicated 
by the column of numbers to the right of the frames. The marking in frames 2, 3, 6 and 7 
etc. indicates that those frames have been registered. The marking is just a way of 
illusfrating that the computer keeps track of which frames have been registered, as a check 
10 to msure that no frames are omitted during the registration procedure of Figure 49 and 50. 
The manner of selecting the other frames, e.g., frame 4 and 5, can be based on a number of 
criteria, such as the order of obtaining the frame, the neighborliness to other frames afready 
registered, and so forth. 

Figure 52 Is an illustration of cumulative regisfration based on the first captured 
15 frame (Fl) as being the base line for all successive registrations. This is essentially the 
technique of Figure 48. Figure 53 illusfrates an alternative registration procedure in which 
each frame in the set of frames is registered to a cumulative registration 3-dimensional 
model of the object, in sequential order, with one iteration of the frame regisfration process. 
This is followed by an updating of the cumulative 3-dimensional model and a repeat of the 
20 registration process with updated values for the transformation matrix [T] for each frame. 
The process continues until, the quaUty values are within acceptable limits, or after a 
predetermined number of iterations have been performed. Still other possibilities for 
cumulative registration exist. The choice of which one to use will depend on the available 
computing resources, the amount of time requfred to perfonn the cumulative registiation for 
25 the technique, and the desired accuracy. 

Figure 54 is a screen shot of a workstation computer (e.g., either a scanning station 
or back office server workstation), showing the available regisfration parameters and 
variables tiiat can be changed to optimize tiie regisfration when performing either a frame to 
frame regisfration or a cumulative regisfration. The parameters may vary widely 
30 depending on the type of object being scanned, the amount of time needed to obtain a result 
from regisfration, the speed at which the scanner is moved relative to the object and the 
amount of overlap, etc. Figure 54 illustrates that the user is able to select and modify the 
registration procedure parameters as they see fit. Two different types of regisfration are 

67 



wo 01/80761 

PCT/US()l/liy69 

indicate here, a W registration, in which the quaUty index ("distance limit") is 250 
microns, and a fine registration, wherein the quaHty index is reduced to 50 microns. The 
distance Ihnit is computed as.the square root of the sum of the squares of the nomial vectors 
divided by the number of pomts in the frame. The tema "stationary count" indicates the 
number of iterations to continue of little or no improvement m the quahiy index is seen. 
The Radius value refers the filter R sho^vn in Figure 45. The convergence fector 0.10 refers 
to the minimum amount of improvement needed between successive frames before a 
stationary count commences. The convergence factor is computed by taking the difference 
of the squares in the quality index of the ith. iteration and the i - 1 th iteration and dividing 
by the square of the quaUty mdex of the ith iteration. 

The number of points to register indicates the minimum amount of overlap m points 
(within boundary R) needed to attempt a regisfration. An "accelerate"' factor is shown, with 
a value of 1.6. This means that the points are moved in the X, Y and Z directions m the 
transformation matrix by an amount of the net normal vector multipUed by the accelerate 
factor. The use of an accelerate factor has been found to reduce the number of iterations 
required to meet the quahty index. 

, The maxhnum iteration count value is a stop value to keep the process from nmning 
into an endless loop. The overly size value is a limit, in terms of mm ^ of the size where 
registration is performed. This serves to screen out stray pomts from the registration 
algorithm. The minimum quota of active points is a minimum amomit of overlap between 
two frames before registration will be attempted, expressed as a fraction of 1. The 
maximum tiiangle size is a filter to filter out triangle sizes where the size of the triangle is 
too large, indicating a stray data point The maxhnal edge length is simply the maximum 
pemiissible length of one side of one of the triangle surfaces. The Maxhnal count of 
unsuccessfiil files is the number of unsuccessfiil sequential registrations before a failure of 
the registration process will be declared. . 

Figure 55 is a screen shot from a worlcstation computer showing a frame to frame 
registration in accordance witii Figure 40 for two fi^es in a set of frames. The various 
parameters shown ux Figure 54 are selected and used m flie frame to frame iteration. In this 
mstance, fi^e 47 is being registered to frame 46. The surface of frame 46 is shown in 
white, frame 47 is shown in dark tones. The left hand side of Figure 55 shows the results 
of each iteration, includmg the nmning time, the number of the iteration, the number of 
overlappmg points, tire overlap between frames (U), expressed as a fraction of 1, the quality 
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index MA, and the -value of the filter R. After 3 iterations, the quaUty index for coarse 
registration was met. The process continued with the fine registration. A series of fine 
registration iterations were performed. Note that the quality index MA improves with each 
registration iteration. 

5 The data firom the last twenty iterations, and the final result, of a registration of 

firame 2 to fi-ame 1 in a typical scan of teeth are shown in Figure 56. After 45 iterations, the 
distance limit of 30 microns was met (MA = 27.686 microns). Note that the graphical 
representation of firame 1 (white) and firame 2 (darker tones) is such that there is essentially 
an equal amount of firame 1 and firame 2 in the picture. This indicates that a "best fit" 
10 between firames 1 and 2 has been achieved. 

D. Segment Registration 

When scanning any object, such as teeth, the situation may arise in which the 
operator of the scanning cannot capture all the surfaces of the object in one scaniiing pass. 
The iriteiruption may be due to the need to physically move the scanner to a location that is 

15 impossible to reach firom one location, the need for the patient to take a break firom the 
scanning, or some other reason. When scanning teeth of a single jaw, the scanning is 
typically performed in two or three different segments. First, one side of the jaw is scanned, 
then the front of the jaw, and then the other side of the jaw. In this situation, there are three 
different segments of the object. All the frames of each segment are registered to each 

20 other, typically using a frame to frame registration. Then the segments are registered to 
each other. After this has been done, a cumulative registration is performed of the entire 
jaw. 

To perfonn the segment registration, th^e must be some way of indicating where at 
least one point in one segment is common to another segment. Segment registration thus 

25 requires some overlap between segments. The scanning workstation provides a mechanism 
to iudicate at least one poiat where two different segments overlap. In the case of the 
scanning of teeth, the operator of the scanner will typically include the canine teeth in scans 
of both sides of the jaw, and m the scan of the front of the teeth. The operator can also be 
instructed to scan these teeth in the. side and front segments. Therefore, the segment 

30 registration proceeds by the user selecting or uidicating a point on the canine teeth to use for 
performing segment registration. A procedure referred to herem as "landmarking" is used 
to select the point used to register segments. It will be understood that a similaf process 
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Will be performed when scanning other types of objects where more than one segment was 
used to completely scan the object 



E. Landmarking 

Figure 57 is a screen shot sho^ving a graphical representation of a three-dimensional model 
of a patient's upper front teeth (segment 1) after a frame to frame registration of this 
segment. The user is applying landmarks to the canine teeth as a preliminary step in 
treatment planning, and as a step in registering overlapping segments of a scanned upper 
jaw relative to each other to calculate a complete model of the upper jaw and associated 
dentition. 

The purpose of the landmarking shown in Figure 57 is to select a point on the canine 
teeth which is common to the front scan and the two side scans. The landmaxldng is also 
done at a point on the labial surface of the teeth that would be a suitable location for 
placement of an orthodontic bracket as part of an appliance to correct a malocclusion. The 
landmarks, are characterized by both a point location and an orientation. To place the 
landmarks, the user chcks on a tooth number, indicated by tire row of numbers 301, and 
drags the cursor witii a mouse to the surface on the canine teeth where they wish to place the 
landmark. They then release the cursor, and the landmark 302 appears on tiie tooth. The 
landmark has an arrow 304 which must point to tiie indsal edge of the tooth. The user can 
rotate the landmark to place the arrow in the proper orientation by simply clicking on tiie 
landmark and turning the mouse one way or the other. As each landmark is placed, a box 
below the tooth number is hightighted as indicated at 306. 

The tootii numbering convention shown in Figure 57 is as follows: tiie first number 
indicates the quadrant of the patient's dentition, witii 1 being upper right, 2 being upper left . 
3 being lower left, 4 being lower right The second nmnber is tiie tooth number witii- 1 
being the incisor. Thus, tiie landmarks are placed at teetii 13 and 23, tiie upper canines. 

Smce tiiese canines overlap tiieir respective side scan, and since tiie X, Y and Z 
coordinates of tiie point on tiie labial surface of tiie tootii where tiie landmaric is placed is 
assigned in tiie computer, it is now possible to register the front segment shown in Figure 57 
to tiie two side segments. This segment registration is now performed. The overlapping 
frames between each segment can be registered to each otiier, or to tiie entire otiier 
segment 
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After se^ent registration is perfonned, a cumulative registration of the entire jaw is 
perfoimed.in accordance with the procedure of Figure 48. After the cumulative 
registration is perfomied, the virtual three-dimensional model of the entire jaw is presented 
to the orthodontist on the monitor in the back office server workstation 28. 
5 Note that if the scanning is done in one pass, e.g., where it is perfomied on a plaster 

model, there is no need for segment registration. The landmarking step can be eliminated in 
that event, although it may nevertheless be perfomied as a step in placing virtual brackets on 
the teeth objects of the virtual model.- 

In planning treatment for the patient, the orthodontist conceptuahzes teeth as 
10 individual teeth objects that can be moved independently of each other to correct the 
patient's malocclusion. Furthermore, orthodontists are trained to make physical models of 
the patient's dentition from an impression, cut the teetii from the model, and then 
individually move the teeth relative to each other to provide a target situation which corrects 
for the malocculsion. Therefore the back office server workstation 28 includes interactive 
15 treatment plamiing software which enables the orthodontist to do this with the virtual three- 
dimensional model of die patient's dentition. In order to do this tireatinent planning, it is 
highly desirable therefore to process the three dimensional model resulting from a 
cumulative registration by separating the teeth from the gums and otiier anatomical 
structure, and presenting the just crowns of the teeth to the orthodontist. This allows virtual 
20 individual teeth objects to be moved independently in three dimensions on tiie computer. 
This process of separation of the teeth from the cumulative registration into individual teetii 
objectsjwplll he ^described next. 

, fhs separation process described below has one fiirther advantage, namely requirmg 
less rieinoVy to represent an individual tooth. Cumulative registration may result in an 
■ ■■25 extremely, large n^bk of poiiits from a large number of frames to rq>resent any ^ven 
tooth. The separation process, as described below, reduces this data set to a single set of 
points that describe a- single surface representing the surface of tiie tooth. Much less 
. memory. is.required. Consequently, the txeatinent planning software can process treatinent 
planning steps for the teeth more quickly. 
30 • ' 



71 



10 



15 



20 



25 



30 



WO 01/80761 

PCT/USOl/1196!) 

F. Separation of Teeth into Tr,H,-.HH.,,, -r^ ^tb Qhie^t. ftr^r^fU ^^^^j^-^ 
Figure 58A-58F are a series of iUustrations showing the generation of an individual' 
tooth model firoxn a scanned tooth. The process will now be explained in detail. 

Figure 58A shows the scanned dentition and ^sociated anatomical structure 
surrounding the tooth 308. This tooth is tooth number 14 in the numbering convention 
shown in figure 57. Tl.e back office server workstation stores a three-dimensional virtual 
template tooth object for each tootii in the maxilla and the mandible. Tl.e template tooth 
310 for tooth number 14 is sho^vn in Figure 58B. The template tooth object 310 is a three- 
dimensional tooth object having a single set of points defining the boundaries of the tooth. 

shown m Figure 58C, the template tooth 310 is positioned approximately in the same 
location m space as the tooth 308. TT.e landmark 302 assists in providing the proper axial 
rotation of the template tooth to have it fit properly with respect to the tootii 308 The 
template tootii is placed at tiie point cloud of tiie dentition according to tiie labial landmark 
302. THe template tooth can be scaled larger or smaller or positioned arbitrarily by the user, 
m order to get a close a position as possible to the point cloud of the dentition 

• As shown in Figure 58D, vectors are drawn from the points on ti.e template t< .tii to 
tiie scanned point cloud of tiie tooth 308. Every ray intersects several surfaces, depending 
on how often tiae respective part of ti.e surface has been covered during scamiing. For each 

vector, a surface is selected. Preferably the smaiu<:i- tr,-o„^i _x- • , 

xci^uiy, me smallest tiiangle surface is selected, since tins 

surface corresponds to an imase taken by tiie scanner n,^ .... 

„ uie scanner when the scanner was positioned in a 

more peipendicular orienMon ,o the dentition resulting in more .ccnrBcy in the 

detennination of the coordinates of that portion of the surfece. As another possibiUty, the 
outennos, surfec. is selected, using a filter to insfre that no extraneous surfaces are used. 
These pomts of the surfeees intersected by aU the vector, are combined as newly generated 
tnangle surfaces and therefore fom, one consistent surface shown in Figure 5SE. ' Then, 
finally, missing parts of the tooth are completed ffom the template toott Tie result is 
shown in Figure 5SF. In a second pass, this generated, object is then used as a template 
tooth, and the steps indicated by Figure 58C - 58F are repeated in an iterative fashion. Tins 
.s done to make sure that the algorithm works eveh if , there are significant differences 
be^een the original template tooth and the scamred point cloud. e.g, a gap in scan data, 
different geometry in the tooth, tte goal is to provide an algorithm tha, does not required a 
closely fitting template tootii object. 
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The final result, an individual three-dimensional virtual tooth object 312, is then 
displayed, to the user, as shown in Figure 59. The result may be displayed on the ■ 
workstation user interface as a three-dimensional superposition of the original data (white) 
. and the separated model of the tooth (darker tones or contrasting color). These tones allow 
5 the user to ascertain whether there is an even distribution of white and daik tones, indicating 
good fit between the scanned tooth 308 and the individual tooth object 312. This step may 
be automated by an algorithm detecting the difference (or the sum of the differences), and 
repeating the process if the difference is too great. 

This process is of course perfoimed for all the teeth. The result is a set of individual 
10 tooth objects for all the teeth in the patient's dentition. The teeth can be displayed either 
alone, or in conjunction with the surrounding anatomical structures such as shown in Figure 
59. 

Some human interaction is used in the embodiment described above in context of 
Figure 58. While the process could be performed for all the teeth in both arches on the 
15 workstation at the orthodontic cUnic, that is not necessary. In particular, since the virtual 
model of the dentition and the terrrplate teeth exist as digital data ui memory, they can be 
transported to a remote location and the task of separation of the dentition mto virtual teeth 
objects could be performed at another location. This has the advantage of not tying up the 
back office workstation or server 28 in the clinic unduly, and requiring less labor at the 
20 clinic. We therefore contemplate that the function could be performed as a service of the 
precision appUance service center 26 of Figure 1, or perhaps even by some other entity or 
service provider equipped witii the necessary computer hardware and software. Once the 
virtual tooth objects are obtained for all the teeth in the dentition, the set of virtual tooth 
objects could be sent over the hitemet back to the clinic for treatment plaianing and other 
25 purposes. It would also be possible for the entity performing the separation of tooth objects 
to also present an initial proposed treatment to the orthodontist (such as a target situation, 
location of brackets, and design of orthodontic archwire), and let the orthodontist take the 
process firom there or simply indicate her approval. 

Separation of teeth firom the virtual model of the dentition could also be performed 
30 automatically using algorithms to detect incisal edges of the teeth, grooves between teeth, 
and grooves indicating the intersection of the gums and the teeth. 

Two types of errors can occur when separation of teeth objects firom other structure 
(e.g., other teeth and gums): 1) the data is selected for a tooth tiiat does not in actuality 
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belong to the tooth, such as gums and adjacent teeth, and 2) data that does belong to the 
tooth is ignored. 

We address the first problem by providing an erase mode on the workstation 
software that is perfomiing the modeling process. In this mode, the user is provided with a 
tool that erases triangle suifeces from the 3-D data, e.g., by highlighting unwanted areas 
with a mouse and clicking an erase icon or other similar technique. As each tooth is 
modeled individually, parts of the data that represent the unwanted data, e.g., data belonging 
to other teeth or gingival tissue, are eliminated from the tooth. This is only a temporary 
process; it is used only to model that tooth and underlying scanned data is preserved. When 
modeling the adjacent tooth, that data is used again. The erasmg process can be performed 
directly on the original scan data. .However, this can be inconvenient since the original scan 
data can consist of a huge overlay of data. 

As an alternative, and more preferred approach, the user works on a tooth model that 
has akeady been created and consists of one sheU of triangles. Thus, the erasing mode 
would be used for example after one iteration of the process of Figure 58A-F. The 
selection and erasing process is much faster. The modeling algorithm calculates the 
surfeces to be deleted from the model in a single pass. The remainder of the iterations of 
the process of Figure 58 can typicaUy be performed without any further erasing. 

As another alternative for elimination of unwanted data, cutter plane tools can be 
provided on the workstation software to assist the user in selection of correct tooth scan 
data. The activation of this feature is shown m Figure 64A. In this technique, two planes 
1000 and 1002 are superimposed on the scanned virtual model 18 of the dentition. The user 
is able to translate or rotate the planes 1000 and 1002 in any orientation m space using 
suitable navigation tools on the workstation 28 user interface. The planes may be in 
different colors. The planes serve as boundaries for the selection of tooth scan data as an 
individual virtual tooth object, or as part of the iterative procedure of Figure 58. All 3-D 
data that is outside of the planes 1000 and 1002 is ignored. The intention of the planes of 
Figure 64A is to simulate the physical process of cutting a physical model of teeth into 
discrete tooth objects. 

Figure 64A indicates that the two planes may not work perfectly since teeth are 
curved or crooked at the contact area between adjacent teeth, and the plane 1000 may in fact 
intersect two different teeth. In Figure 64A, the area in region 1004 indicates where some 
triangle surfaces from the adjacent toofe 30SA are included in the region between the two 
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planes 1000. and 1002. These parts of tooth 308A can be manually erased by the erase 
mode feature described above. 

Anotiler possible method for separation of the teeth, without iQcluding extraneous 
anatomical .stmctures, involves allowing the user to click with a mouse multiple points on 
5 the surface of the tooth where the tooth intersects adjacent anatomical structures. In Figure 
64B, the-user has highlighted areas 1006 on the tooth 308 where the tooth intersects giagival 
. tissue. As each area is highlighted and selected (e.g., by a click of the mouse), the software 
records the coordmates of points associated with these areas 1006. Then, a series of planes 
are constructed -Which connect these points (or surfaces) together. These planes 1008 are 

10 mdicated by the hatched area of Figures 64C and 64D. The planes 1008 serve the same 
functions as the cutter planes 1000 and 1002 of Figure 64A; i.e., they define a boundary 
sepai^ting^-the tooth from associated non-tooth anatomical structures, such as gums and 
other teeth. Depending on the anatomy of the patient, it may be necessary to highlight 
closely-spaced areas, as shown in Figure 64B, so that the planes 1008 match the contours of 

15 the gum and tooth. 

Referring now to the second problem, the tooth separation process of Figure 58A-F 
can be forced to use proper data that would otherwise be ignored- Specifically, the xiser 
clicks certain areas where original scan data has been wrongfully ignored. Clicking on the 
area forces the modeling algorithm to pick original data points from the scan including the 

20 selected areas. For example, region 1010 in Figure 64D has scan data associated with it, but 
such data was ignored in a frame to frame registration process. The user highlights this 
area and points for those areas are filled in from the original scan data for pertinent frames 
covering this area. 

To allow for a safe operation of this user interaction, the modeling algorithm will 
25 iQtemaily mark or classify each generated point in the virtual tooth model as being based on 
scan data (true points), or if it has been constructed by the algorithm due to the lack of data 
(artificial points, suppUed by the template tooth 310 in Figure 58B). A lack of data will 
always occur in the spaces between teeth since the scaimer cannot usually capture images of 
the gaps between teeth effectively. A lack of data can also occur due to unproper scaiming. 
30 The lack of data can be cured to a certain extent by the modeling algorithm of Figure 58, 
with the lack of data suppUed by the template tooth, e.g., in the gaps between teeth, and 
adapting this template tooth to the scanned dentition as described above. Artificial points 
can be-m'arked as such and displayed m a different color or using lighter or darker tones. 
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The m^puMons of the user descnbed above for wrong&Uy ignored data wfll have effect 

only on the artificial surfaces. 

Missing data or gaps from the tooth Sean can also be filled from the template tooth 
usmg a sunple intetpolatjon procedure, such as sho™ in Figure 65. The example of Figure 
6= wtU be gtven m one dimension (Z); a similar process occurs in the other dimensions X 
and Y. As shown, the scan data for the tooth 308 includes a gap indicated at 1020. The 
template tooth includes a ^e 1022 that corresponds to this gap. To fill in d.e surface, 
the dtstanoe in mm is detemrined between the point at the left hand edge of the gap 1024 
and the template sur&ce 310 in the Z direction (0.6 mm). The same is done for the right 
hand edge 1028 (0.9 mm). A mid point is chosen 1026 and the values are averaged to arrive 
atthedistaneeforpotat 1026 (0,75 mm) This process is repeated fcr intemtediate points as 
mdtcated. After a suitable number of interpolations, the points m the template tooth 310 are 
moved m the Z direction by the measured or calculated amom^ts, hete 0.6 mm, 0.68 mm. 
0.7= mm. 0.83 mm. 0.9 mm. These points are now comiected by lines to fom. triangle 
snifeces to complete the smftce of the tooth 308. 

The tooth model, once created, can be modified to simulate various treataients that 
may be made on 4e tooth, such as interproximal teducrion, or grinding portions of the tooth 
away, by u«ng the erase mode, cUpping planes similar to the planes 1000 and 1002 of 
Figure 64A, or some other convenient means. 

The Bbrary of standardized template teeth described above is based on standard 
tooth models for each teeth in one possible embodiment The library described above could 
be augmented to include a library of teeth based on the ethnicity, sex, age, or other factors. 
For example, the Bbrary could consist of one Ubrary of template teett for Hispatncs, another 
hbrary of template teeth for Orientals, a thini Bbrary of template teeth for Caucasians, etc. 
These hbranes could be stwUed with the tteatment plamung software, obtained from a 
<iurd party, or even created from patient scans. For example, after each scan using the 
present sy^ the software cuts dte teeth mto individual tooth objects using the process 
descnbed above and stores the teeth in memory, along with identiftdng indicia such as the 
race, sex and age of the patient. Over time, a large coUection of virtual tooth objects for 
vanous types of patients will be obtamed. These virtual teeth can be registered to each other 
for a gtven ethnic type, on a tooth by tooth basis, to result in a new Ubraty of template teeth 
for a given ethic group, which are stored in memory. The user can thereafter identic for 
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the software which Ubraiy of template teeth to use for a new patient based on the relevant 
criteria such as ethnicity. 

Alternatively, in soihe situations it may be desirable to use a contralateral tooth as 
tiie template tooth, and not a template tooth from the library of template teeth. In this 
5 situation, a tooth object of one of the contralateral teeth is obtained from a template tooth in 
a Ubrary and the cumulative registration scan. Then, the tooth object of flie contralateral 
tooth is obtained by using the contralateral tooth as the template tooth and the cumulative 
registration scan as explained above. 

In some situations, the template tooth may need to be modified. A few example of 
10 such situations are a partially erupted molar, a tooth that has been chipped, or a tooth that 
has been the subject of extensive previous dental work. Since the template tooth exists as a 
mathematical' model, it can be displayed on the user interface and modified. The 
modification can be made using suitable navigation software and a cUpping or erasmg 
feature to delete part of the model. One way is providing a clipping plane feature, by which 
15 a plane mtersects the template tooth in a orientation and location defined by the user usmg 
suitable navigation tools. The portion of the template tooth on one side of the plane is 
deleted. The user positions the plane at the desired location on the template tooth to 
roughly match the anatomical structure of the tooth m question. This process will result m a 
smooth execution of the tooth separation algorithm and resuU in a virtual tooth model that 
20 substantially exactly matches the structure of the patient' s tooth. 

The virtual tooth model may be extended beyond merely the crowns of the teeth. 
For example, a hbraiy of standardized visual root templates for each of the teeth may be 
stored in the memory of flie workstation. As individual virtual models of each tooth are 
created, the standardized root for that tooth are matched up with the virtual tooth model to 
25 thereby created an individual virtual tooth model of tiie entire tooth. 

This process can be extended to templates of virtual gum tissue. On one hand, after 
separation of the indi\ddual virtual tooth models from the gum tissue the remaining portion 
of the scan data depicts the gum tissue (or at least a portion of the gum tissue, depending on 
the tissue scanned). This gum tissue may be substantiaUy incomplete. The incomplete 
30 portions can be supphed by a template of vhtual gum tissue, e.g., gums for an entire arch. 
The template of virtual gum tissue can be scaled up or down or modified as may be 
necessary to fit the anatomical struchire of the patient A registration of the template gum 
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to the scanned gingival tissue enables a complete three-dimensional virtual model of the 
gums to be created. 

TOs process caa be extended to template bones, both mandible and maxilla. The 
goal here is to create a three-dimensional virtual model of fte patient's mandible ormaxilla 
A vntual three-dimensional model of the mandible and maxilla can be obtained from a 
vanety of sources, including CAT scan data, or skeletal specimens. The models are stored 
m the memoo' of the workstation. The virtual template mandible and maxilla are then 
expanded, contracted, or otherwise modified to fix to the patient's anatomy. X-rays of the 
patten, may assist in the process. When the modified virtual template mandible and maxilla 
are created, the virtual teeth, gums and or roots can be displayed together flie maxilla 
or mandible, either alone or together with the orthodontic appUance. 

Ute concept of template virtual objecte can be extended to virtual template crowns, 
and the scamung fean^ and user inte*ce on the workstation extended to simulation of 
vntual crowns and adaptation of virtual crowns to virtual prepared teeth. For example, a 
prepared toott is scamied as described herein and represented as a virtual three-dimensional 
model. A template viffltal three-dimensional crown for the toofl. (and typically for aU 32 
teeth) ,s stored in memoty on the workstation or accessed fiom a database. The shape of 
a,e v«ual template crown is reduced or ad^ted to form fi, the prepared tooth where crown 
and prepared tooth surftces meet. The sh^e of the cusps of the ctown can be obtained 
ftom the surface configuration of the opposing tooth and adjacent teeth, or from the surface 
configuration of a contralateral tooth that is also scamted and represented a virtual tooth 
object. 

Once the crown shape has been anived a, on the woAstadon, it can be exported as a 
3-D crown object file to a remote location such as a lab for manufacture of the crown. For 
«ample, the S-D crown object file is fed to a stereoli&ogiaphy machine and a physical 
model of the crown is made. A mold is made from the physical model. Tie crown is 
fotmed in the mold fiom gold or porcelain made using fl.e lost wax technique. 
Altentatively, the 3D crown object file can be supplied to a CNC milling machine for 
machmmg a crown. Such cn>wns could be made flxrn, any suitable ceramic or metalHo 
matenal, mcluding stamless steel. Tlis process represents a substantial savings of time and 
labor m manufi«:turing crowns. The typical cn,wn prepatafion process is a manual process. 

The concept of virtual template tooth objects and user manipulation of tooth objects 
on a computer can also be used in fl« field of denudes. Traditionally, an impression is 
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taken of the gums and associated bony alveolar structures and these anatomical structures 
are cast in plastic or wax. Pre-fonned teeth are set into the wax in a desired occlusion. The 
dentures are cast in acrylic using the lost wax technique. This process can be automated 
using the scanning methods described herein and usiag virtual three-dimensional template 
5 teeth. First, the gums and associated anatomical stractures are scanned and represented as a 
three-dimensional virtual model on the workstation. Then, vktual template teeth are 
retrieved from memory. The template teeth are sized up or down as necessary to conform to 
the archform represented by the virtual model of the gums. The virtual template teeth are 
then placed on the archform. At this point, a three-dimensional virtual model of the teeth, 
10 gums and associated anatomical structures is represented in the workstation memory as a 
three-dimensional virtual object. This digital object can be exported anywhere, such as to a 
remote location where dentures are manufectured. From this object, a denture can be 
manufactured from a variety of techniques, including milling and casting. For example, a 
stereolithographic physical model of the dentition and/or gums can be made and a denture 
15 cast in a mold obtained from the physical model using the lost wax technique. 

The virtual template teeth can also be used in forensic dentistry, i.e., reconstruction 
of the identity of a victim from teeth. As an example, a jaw contaming some teeth can be 
scanned as described above and represented as a three-dimensional virtual object. Missmg 
teeth can be reconstructed by importing virtual template teeth and placing them on the 
20 virtual object. The virtual template teeth may be based on age or ethnicity if such 
inforaiation is known. Contra-lateral teeth can be constructed by using existing scanned 
teeth as the template tooth and placing the scanned tooth in the contralateral position. 
Eventually, a complete virtual representation of the dentition can be obtained and viewed on 
the workstation. The shape of the face of the victhn can be reconstracted by adding 
25 template virtual objects comprising soft tissue, gums. Hps, cheeks, skin, hair, etc., and 
modifying the template objects using navigational tools based on the three-dimensional 
object or other information known about the victim. 

Another example of using template teeth is for purposes of diagnosis and detection 
of tooth wearing, e.g., due to bruxism. M this example, the original scan taken of the 
30 patient is converted into a three-dimensional virtual model. The individual teeth are 
optically separated into virtual three-dunensional tooth objects as described above. Either 
this original vutual model of the entire dentition or the set of vfrtual three-dimensional tooth 
objects can be considered as a template. Over the course of time, the dentition is scanned 
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again periodicaUy and converted into a three-dimensional virtual model as described above. 
The mdividual teeth (or the dentition as a whole) is compared to the template to identify 
differences due to wearing of teeth. This can be perfonned by overlaying the tvvo models, 
each m a different color or tones, and. visuaUy detecting where tooth surfaces were present 

initially but are not oresenf m , , . , 

, c vixLiuu moaei. Alternatively, measuring tools can 

be provided on the user interface to measure the height of the tooth or other distances that 
may be mdxcative of wear, and numerical values immediately presented to the user on the 
user mterface. These measurements can be compared with measurements made of the 
template. Now. tooth wear can be quantified precisely. 

As yet another possibility, individual tooth objects are obtained from the original 
scan of the patient These tooth objects are stored in the memoiy. In the case of a loss of 
the patient's tooth due to an accident or due to an extraction, the virtual tooth objects 
provide a precise template for manufacture of a replacement tooth. The replacement tooth 
could be manufactured for example using the stereohthograpy and lost wax techniques 
referred to above. 

The creation of virtual tooth models allows virtual brackets to be virtuaUy bonded to 
mdzvidual virtual teeth. The virtual brackets are obtained from a 3D CAD model of the 
bracket obtained from the manufacturer of the bracket. Alternatively, the brackets could 
be scamied and virtual bracket models obtained from registration of the scan frames into a 
vntual three dimensional model of the bracket. In either event, the virtual brackets are 
stored m memory and later accessed from the user interface of the orthodontic workstation. 
For example, the virtual brackets are placed on the teeth at the location of the landmarks and 
men moved by flie user accessing suitable navigational tools provided by the user interface. 

The virtual bonding of the brackets is merely a superposition of the virtual bracket 
onto the virtual smiace of the tooth. Since both the bracket and the tooth are separate and 
mdependent virtual objects, they can be moved freely relative to each o&er. for example to 
optmnze the position of the bracket Preferably, the treatment plamiing software allows 
the user to interactively position the brackets along any combination of X, Y and Z 
dnections, as wells as rotation about three orthogonal rotational axes. In one possible 
embodmient. the bracket placement correction is made by the user performing the foUowing 
steps: 
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1) navigating through the treatment planning software until the virtual model of the 
dentition aiid the virtual brackets are displayed (this can be either the target 
situation or the malocclusion); 

2) selecting a bracket for movement by either clicking on the bracket or selecting a 
5 bracket number from a drop-down menu; 

3) accessing navigational controls for the bracket, such as by cUckdng on an icon 
that displays navigational controls for moving virtual objects such as brackets; 

4) allowmg the user to select either move the teeth with the bracket or move the 
bracket freely in three dimensions; and 

10 5) usmg the navigational controls to move the brackets in three dimensions as 

desired. 

If the bracket is moved independent of the tooth model, when the user is finished with the 
movement of the bracket the virtual tooth is moved to the location of the bracket. Bonding 
corrections for bonding the bracket to the tooth are updated. The bracket is then virtually 
15 bonded to the tooth. This process can be performed for each tooth. The result is that the 
orthodontist customized the placement of virtual brackets to the teeth. The archwire, which 
passes through the bracket slots, wiU have the required bends to move the teeth to the 
desired target situation regardless of the positioning of the brackets on the teeth. 

The combination of the displayed set of virtual orthodontic brackets, together with 
20 the virtual orthodontic archwire, thus presents to the user a customized virtual orthodontic 
appliance. The virtue of the customized virtual orthodontic jqjpliance is that it can be 
studied, modified, shared between two computers, and transported electronically over a 
communications medium for febrication of the orthodontic appliance. The treatment 
planning software is essentially a specialized CAD/CAM system that allows the design of 
25 virtiiaUy any configuration of tooth objects, bracket objects, wire objects and other 
appUances and objects. Because these objects exist as independent mathematical objects, 
they can be selectively displayed together or alone. For example, the treatment planning 
software displays an icon or button on the user interface that allows the user to select or 
deselect the teeth, wires, brackets or virtual objects or appHances, as desired. For example, 
30 the teeth and archwire can be displayed together with the brackets deleted from the user 
interface. The orthodontist can then select an individual tooth object, move it in three 
dimensions, and the movement of the tooth carried over to a repositioning of the bracket m 
three dimensions and a changing of the shape of the archwire. 
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Furthermore, while the above process of creation of tooth models has been described 
in conjunction with the scan data from the hand-held scanner, this is not required. The 
separation of tooth objects can be performed with any three-dimensional model of the teeth, 
regardless of how the three-dimensional model is obtained. The three-dimensional model 
could be acquired from a CT scan, a laser scan from a plaster impression, or otherwise. 



Part 4. Treatment Planning 
Introduction 

Treatment planning software is provided on the workstation of ffae orthodontic 
clinic, and possibly at other remote locations such as the precision appliance center of 
Figure 1. The freatment planning software can be considered an interactive, computer- 
based computer aided design and computer aided manufacturing (CAD/CAM) s" ^tem for 
orthodontics. The apparatus is highly interactive, in that it provides the orthodontist with 
the opportunity to both observe and analyze the current stage of the patient's condition and 
develop and specify a target or deshred stage. A shortest direct path of tooth movement to 
the target stage can also be determined. Further, the apparatus provides for simulation of 
tooth movement between current and target stages. 

hi its broader aspects, the apparatus comprises a workstation having a processing 
unit and a display, and a memory storing a virtual, complete three-dimensional model 
representing the dentition of a patient. The virtual three-dhnensional model can be obtained 
from one of several possible sources; m the preferred embodiment it is arrived at from a 
scanning of the dentition. The ^paratus further includes software executable by the 
processing unit that accesses the model and displays the model on the display of the 
workstation. The software further includes navigation tools, e.g., typed commands, icons 
and/or graphical devices si^erimposed on the displayed model, that enables a user to 
manipulate the model on the display and simulate the movement of at least one tooth in the 
model relative , to other teeth in the model in three-dimensional space, and quantify the 
amount of movement precisely. This simulation can be used, for example, to design a 
particular target situation for the patient 

The development of a unique target situation for the patient has utility in a variety of 
different orthodontic appUances, including an approach based on off-the-shelf or generic 
brackets and a custom orthodontic archwire. The scope of the invention is sufficient to 
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encompass other types of appUances, such as an approach based on customized brackets, 
retainers, or the removable aUgning devices mentioned earher. In a bracket embodiment, 
the memory contains a Ubrary of virtual, three-dimensional orthodontic brackets. The 
software permits a user to access the virtual brackets through a suitable screen display, and 
5. place the virtual brackets on the virtual model of flie dentition of the patient This bracket 
bonding position can be customized on a tooth by tooth basis to suit individual patient 
anatomy. Because die tooth models, brackets and archwire are mdividual objects, and 
stored as such in memory, die treatment planning apparatus can simultaneously display the 
virtual brackets, die archwire and the virtual model of die dentition, or some lesser 
10 combination, such as just the brackets, just the dentition, or the brackets and die archwire 
but not die teetii. The same holds tine witii other appUance systems. 

hi a preferred embodiment, the virtual model of teedi comprises a set of virtual, 
individual diree-dimensional toofli objects. A mefliod of obtaining the tooth objects fiom a 
scan of teeth, and obtaming other virtual objects of associated anatomical stiiictinres, e.g., 
15 gums, roots and bone is described. When die teeth are separated from each otiier and from 
die gums, tiiey can be individually manipulated. Thus, die individual toodi objects can be 
mdividually selected and moved relative to odier teedi m the set of virtual toodi objects. 
This feature permits individual, customized toodi positioning on a tootii by tooth basis. 
These positioning can be in terms or angular rotation about tiiree axis, or translation m 
20 transverse, sagittal or coronal planes. Additionally, various measurement features are 
provided for quantifying the amount of movement. 

:One pf,the primary tools m the Ireatinent planning apparatus is the selection and 
customization or a desired or target aichform. Again, because die teetii are. individual tootii 
objects, diey caS be moved independently of each otiier to define an ideal arch. This 
. ■ ■ 25 . development of the target, archform could be calculated using interpolation or cubic spline 
■ ■ algorithms. Alternatively, it can be customized by die user specifying a type of archform 
(e.g, Rodi), and die tootii are moved onto diat archform or some modification of tiiat 
archform. .. The archform can be shaped to meet die anatomical constraints of the patient. 
After die initial "archform is designed, die user can again position die teedi on the archform 
30 as diey deem appropriate on a tootii by toodi basis. The treatment planning software dius 
enables die movement of die virtual toodi objects onto an archform which may represent, at 
least in part, a proposed treatment objective for the patient. 
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Numerous otiier featwes are possible with the treatment planning software, 
including movement of the teeth with respect to the other teeth in the archfoim, changing 
the position of the virtual brackets and the teeth with respect to each other, or opposing teeth 
with respect ot the selected archfonn. Custom archwire bends can be simulated to provide 
additional corrections. Bonding corrections at the bracket-tooth interface are also possible. 

In another aspect of the invention, a method is provided for digital treatment 
plamiing for an orthodontic patient on a workstation having a processing unit, a user 
interface including a display and software executable by the processing unit. The method 
comprises the steps of obtaining and storing a three-dimensional virtual model of teeth 
representing the dentition of the patient in a current or observed situation. The virtual model 
is displayed on the display. The method further includes the step of moving the position of 
teeth in the virtual model relative to each other so as to place the teeth of the virtual model 
into a target situation and displaying the virtual model with the teeth moved to the target 
situation to the iiser. Parameters for an orthodontic appliance to move the patient's teeth 
15 from the current situation to the target situation can be derived from the virtual model and 
the target situation. For example, if virtual brackets are placed on the teeth, their location 
in the target situation can dictate the design of an archwire to move the teeth to the target 
situation. 

In a preferred embodiment, the method includes the step of providing screen 
displays on die display enabling a user of the workstation to operate the user interface so as 
to place virtual three-dimension^ objects representing orthodontic appUances, e.g., 
brackets, onto the surface of teeth in the virtual model. A Ubrary of the virtual brackets can 
be stored in memory and a landmarking procedure used to place the brackets on the teeth at 
the desired location. Anatomical considerations may'dictate movement of the brackets from 
their originally selected position to a new position. Accordingly, the software provides 
navigational tools enabhng a user to change the position of the brackets relative to the teeth. 

The freatment planning system is based on individual tooth objects which can be 
moved to any position in three dimensional space. They can be moved in several ways - by 
direct user specified movement, and by adding an object comprising an orthodontic 
appUance and changing the configuration of the apphance to cause the teeth to move. For 
example brackets can be virtually bonded to the teeth and the. position of the brackets 
changed in three dimensions to move the teeth. Alternatively, an archwire shape can be 
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defined which fits into the slots i the brackets. Movement of the archwire can be simulated, 
resulting ii a simulation; of tooth movement. 

The treatment planning software includes features enabling more accurate diagnosis. 
For one thing, the virtual model of the dentition can be manipulated in three dimensions at 
5 wiU, resulting in complete visual assessment of the model. Measurement tools are also 
provided by which the orthodontist can deteimine the distance between any two points on 
the model. This allows the user to quantify the patient's morphology both at initial and at 
target stages. Thus, treatment progress, proposed changes in appHance design, or tooth 
movement can be quantified precisely. By measuring the differences and changes in 
10 morphology during the care cycle, the orthodontist can quickly and accurately assess patient 
treatment. Changes in treatment can be made early on. The result is shorter treatment times 
(and the ability for the orthodontist to service more patients per year). 

The treatment planning system incorporates virtual objects comprising orthodontic 
appliances that may be used to treat the patient. The invention provides for desi^ of 
15 complete apphance systems and simulation of various treatment scenarios prior to initiation 
of treatment 

A bite registration scan is obtained firom the patient to spatially correlate the scans 
ofthe upper and lowerjaws when the dentition is clenched. This scan is used to provide a 
registration ofthe upper and lower jaw to determine the correct relative position. This bite 
20 registration scan is usually needed at the beginning of treatment to set the relation between 

the upper and lowerjaws. 

Landmarks such as shown in Figure 57 are then placed on the labial surfaces of all 
the teeth. The illustrated embodiment places landmarks manually, but this process could be 
automated. The landmarks are placed initially on the molars and the front teeth, and an 
25 estimated position for the landmarks on the other teeth can be made, such as in the same 
plane, based on the relative position of the landmark with respect to the gingival tissue and 
incisal edge ofthe tooth, or other factors. 

The landmarks are placed at the location where the orthodontist expects to place an 
orthodontic bracket to correct the malocclusion. The bracket shape is shown on the 
30 monitor 30 (Figure 1). Three-dimensional templates for a variety of commercially 
available brackets are stored in memory and the software asks the orthodontist to select a 
particular manufacturer and style of bracket to use with the patient. Thus, as the landmarks 
are placed, virtual brackets appear in Ihe computer model on the labial surfaces of the teeth 
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where the orthodontist desires to place the brackets. The orthodontist can move the bracket 
position dg^ending on the type offerees the orthodontist wishes to create on teeth to correct 
the malocclusion. 

Figure 60 is a screen shot from an orthodontic workstation showing the computer 
model of the patient's teeth objects 312 positioned in a target or desired condition. The 
Illustration is tiie result of the user selecting an archform for the patient from a known type 
of archform (e.g.. Roth), and the computer placing the teeth along the arch selected by the 
user. This is executed by placing the virtiial brackets the orthodontist placed on the teeth 
along the curve selected by the orthodontist The br^kets are omitted from Figure 60. but 
. are shown in Figure 61. The software allows the orthodontist to change many variables in 
. the target sitiiation, shnply by entering new values in the shde line area 320 of the screen 
display, by mouse operation of up and down arrows to scroll through available values, or by 
mouse operation of a bar to change the values, or other similar technique. Figure 60 shows 
some of the parameters by which the orthodontist can adjust the shape of the arch, the 
distance between the teeth, the distance between the molars, and other parameters, so as to 
provide a unique and customized target situation for the patient 

Figure 61 is another screen shot showing the computer model of the patient's teeth 
m a target sitiiation, also showing the nmnexous parameters available to the orthodontist to 
customize the tooth position, orientation, angulation, torque, and other parameters on a tooth 
by tooth basis for the target archform. Virtual brackets 322 are positioned on the tooth 
objects 312 at the location where the user placed the landmarks. A virtual archwire 324 
passes through the slots in each vutual bracket 

Figure 62 is another screen shot showing a front view of the target sitiiation and 
additional parameters available to the orthodontist for simulating the movement and 
positioning of teeth relative to each other in planning treatinent for the patient For 
example, in Figure 62, the cursor is moved onto the virtual tooth 41 (in the tooth nmnbering 
convention) and the mouse is clicked. Tooth 41 is then highUghted. If the orthodontist 
wants to exfract that tooth, they then cUck on the box 22 to extract the tooth in the 
smiulation. Alternatively, tooth 41 could be nutated about any of tiiree axis of rotation, 
moved m the X, Y or Z direction, or a larger or smaller gap could be created between teeth. 

Figure 63 shows the target sitiiation for the upper arch, with the virtual brackets 322 
m place. The orthodontist can adjust the bracket 322 position, archwire shape 324, or tooth 
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312 position, on a tooth by tooth basis to thereby optimize treatment planning for the 
patient.- 

The result- of the treatment planning is the generation of a set of bracket placement 
positions and the display on the monitor of the shape of a customized orthodontic archwire 
5 to treat the malocclusion. Information as to the location of the brackets, the three- 
dimensional model of the malocclusion, the three dimensional model of the target situation, 
and the type of archwire. to use are sent to the precision appUance center 26 of Figure 1. A 
customized orthodontic archwire is manufactured in accordance with the bracket location 
■ and type and the target situation for the patient. Additionally, a transfer ti-ay is 
10 manufectured to assist the orthodontist to place the brackets at the proper location. The 
transfer tray, brackets and archwire are shipped to the orthodontist's clinic 22. The 
orthodontic appliance is then applied to the patient and treatment commences. 

Because the hand-held scanner, allows for scans of the dentition in a matter of 
minutes, the scanner becomes an important tool in monitoring treatment As the treatment 
15 progresses, the movement and position of the teeth during treatment can be quamtified with 
a high degree of precision. The orthodontist can discern during treatment that corrections 
in the wire need to be made, for example due to biological influences affecting tooth 
movement. The treatment planning software on the workstation displays the current 
situation, and also the target situation. A new customized archwire is designed on the 
20 computer. The relevant information for making the new archwire is sent to the precision 
appUance service center and a new archwire is manufactured and shipped to the clmic. 

Monitoring scans are taken during treatment to measure and quantify progress and 
detect deviations from the expected treatment. Since each of the tooth objects is ahready 
stored, the monitoring scan need not be of the entire dentition, but rather needs to only be of 
25 one surface, such as the occlusal surfece, or the lingual surface, or some combination of the 
two. A bite scan with the teeth in a clenched condition is also taken to get the cuirent upper 
and lower relation. The position of the rest of the teeth is obtained from the virtual tooth 
objects 312 of die patient's teeth (Figure 58F). After die monitoring scans are performed, a 
registiration is performed of die scanned data relative to the tooth models to complete the 
30 teeth and derive a current virtual model of the patient at that point in treatinent. Study of 
this virtual liiodel and comparison to the target virtual model of the teeth aad the virtual 
model at die beginning of tireatinent can mdicate whether progress is as anticipated or if 
additional correction to the orthodontic ^pUance need to be made. These corrections will 
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typically be carried out as wire changes in an archwe orthodontic treatment regime, in 
which new bends are placed in the orthodontic archwire. 



Initial virtu al brack-et placement 

With the individual teeth now cut from the three-dimensional model of the dentition 
and represented as tooth objects, they can be moved relative to each other in three 
dmiensions. Since orthodontics assumes that a bracket is fixedly bonded to a tooth, by 
movmg the bracket one moves the tooth. The next step in the pn^cess is thus selecting an 
initial location to bond the brackets to the tooth. As noted below, this initial location can 
be adjusted by the treatment plamiing software. The spatial location of the surfeces of the 
bracket and the surfaces of the corresponding tooth are known. Collision avoidance 
algonthms are used to keep the bracket positioned on the surface of the tooth and prevent 
the virtual bracket from entering the tooth itself, a clinically midesirable result. The user is 
able to move the bracket mdependently of the tooth by activating an icon (such as one 
shaped like a magnet to signify the mating of the bracket to the tooth). When the bracket is 
moved to the new location, the tooth matches up to the surface of the bracket. 

The brackets are represented in the software as virtual three-dimensional objects, and 
the surface of aU the brackets and the teeth are known in three dimensional spatial 
coordmates. Accordingly, collision detection algorithms are employed to detect when 
smiulated tooth or bracket movement would result in a coUision between brackets and teeth. 
Smnlar collision algorithms are provided to prevent the adhesion surfece of the bracket from 
nugrating into the body of the virtual tooth object and to keep the brackets located on the 
surface of the teeth. IF the user wishes to move the location of the brackets, the movement 
of the teeth follows the movement of the bmcket. Also, again since the bracket is a three- 
dmiensional virtual object with known spatial coordinates, the user is provided with a tool 
(such as an icon) which when activated allows the user to move the bracket about one plane 
or axis, and freeze the movement in the other directions. 

Initial virtual bracket placement is done as follows. Landmarks 302 sucH as shown 
ui Figure 57 are placed on the labial surfaces of aU the teeth. The landmarks are placed at 
the location where the orthodontist expects to place an orthodontic bracket to correct the 
malocclusion. Hie bracket shape is shown on the monitor. Three-dimensional templates 
for a vanety of commerciaUy available brackets are stored in memory and the software asks 
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the orthodontist to select a particular manufacturer and style of bracket to use with the 
patient. Thus, as the landmarks 302 are placed, virtual brackets appear in the computer 
model on the labial surfaces of the teeth where the orthodontist desires to place the brackets. 
The orthodontist can move the bracket position depending on the type of forces the 

5 orthodontist wishes to create on teeth to correct the malocclusion. Because the brackets are 
individual objects and stored in memory, when they are placed on the surface of virtual 
teeth complete position information is known in three dimensions. As such, the brackets can 
be displayed either alone, or in conjunction with teeth, or hidden from view, by means of 
appropriate user specified commands on the user interface. For example, the screen display 

10 showing the target or current stage can have an icon indicating hide brackets, or display 
brackets, and activating the icon causes the brackets to be hid or displayed. The same is true 
for other virtual objects that exist independently of other objects, such as tooth models and 
thearchwire. 

With the teeth now separated into individual tooth objects, and the orthodontist can 
15 now view the currCTit target stage, custom design a target situation for the patient, and 
design the appliance to treat the malocclusion. These aspects will now be described m 
further detail. 



20 



Viewing the observed f current^ stage 



Figure 69 is a screen shot showing a three-dimensional model 18 of a malocclusion, 
showing the teeth 312 in both the upper and lower arches 326 and 328, respectively. The 
screen 330 includes a row of icons 332 across the upper portion of the display, which are 
associated with various tools available to the user to view the dentition, virtual brackets, and 
25 current and target archforms. The lower portion 334 of the screen includes a set of tabs 336 
that are accessed in various aspects of treatment planning. These tabs 336 include a patient 
tab 338, which accesses the screen of Figure 66. A Iknits tab 340 allows a user to 
breakdown the tooth movement between observed and target stages into stages, such as 30 
percent, 50 percent and 75 percent, and display the tooth positions for both arches at these 
30 positions. A differences tab 342 quantifies the differences (in terms of translation and 
rotation) between the observed and target stages for each tooth. The space management tab 
344 permits the user to simulate extraction of one or more teeth and adjust the spacing 
between teeth in either arch. A bonding correction tab 346 allows for adjustment of tooth 
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position to be realized via bonding corrections. The technique tab 348 aUows the user to 
select a bracket prescription and default settings for bracket height (distance from bracket 
slot to mcxsal edge of tooth). The tab also displays the parameters for the bracket 
prescnption chosen by the user. The upper/lower (U/L) relations tab 327, selected in the 
screen shot of Figure 69, allows the user to modify the relation of the upper and lower jaws, 
by both translation in three axes (transversal, sagittal and vertical directions) and by rotation 
about these axes. The user manually enters values in the field 350 to change any parameter, 
and the change is immediately reflected in the view of the model of the dentition. 

The tabs also include a bracket ofifeet tab 352 that allows a user to reposition the 
bracket on a tooth and specifies numerical values for each bracket placement modification. 
A brackets tab 354 aUows a user to enter information as to the type or manufacturer of 
brackets for each tooth in the both arches. 

A finlher "moiphing" tab could be provided which would animate the movement of 
the teeth firom malocclusion to target situations based on treatment steps or limits defined by 
the user (explained in fijrther detail below). 

The screen shot of Figure 69 also includes a region 356 that allows the user to 
navigate between views of the observed stage and views of the target stage. Here, the user 
has higWighted or selected both arches in the observed stage, so the screen display shows 
the model of the dentition in the current or observed stage. 

Referring to Figure 67, the treatment planning software preferably displays a 
plurality of icons 331, not all of which are shown in Figure 69, to enable the user to quickly 
and easily view the three dimensional model in various standard perspectives. For example, 
the rcons 331 of Figure 67 include icons 333, 335, 337 and 339 for viewing the dentition in 
top, bottom, right hand side and left hand side views, respectively. An icon 341 is provided 
which zooms in or out. Icons 343 and 345 allow the user to select for viewing just the upper 
or lower arches, respectively, including virtual teeth, virtual brackets and virtual archwire. 
The aeon 347 allows the user to show or hide the virtual dentition, excluding brackets and 
archwures. An icon 349 allows the user to select or deselect the virtual brackets. A marker 
icon 341 IS used for measurement fimctions (described below) and an object navigation icon 
353 IS used for manipulating any of the virtual objects on the screen. 

When positioning multiple objects in the three-dimensional view, such as shown in 
Figure 69, the camera navigation icons of Figure 67 move all the elements together. As 
shown in Figure 70, the initial placement of the virtual brackets 400 can be displayed along 
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with the teeth. Further, the camera navigational tools allow the user to zoom in an zoom 
out in any desired degree. However, the vhtual teeth 312 and virtual brackets 400 are 
individual three-dimensional objects which can be selected and moved independently. One 
way of moving objects is by entering new positional values (e.g, in terms of mm of 
5 displacement or angle of rotation, as described later). Another method provided by the 
software is using object navigational controls, activated by clicking the icon 353 or by 
accessing the function via a tools menu. The object navigation controls allow the user to 
move the object based on orthodontic judgment and visual feedback. The amount of 
movement is stored and can be displayed using numerical position information. As will be 
10 discussed in further detail below, the bracket position can be individuaUy adjusted on a 
tooth by tooth basis. Furthermore, the camera navigation icons permit navigation of the 
archfonns (i.e., the teeth placed on some selected archform), navigation of the brackets, or 
navigation of the archwire. 

The object navigation tools first require an object (e.g.,- tooth, bracket, archwire, etc.) 
15 to be selected. To select and deselect any object displayed on the screen, the user places the 
cursor over the object and double clicks the mouse. The selected object is highlighted m a 
separate color. Additional objects are selected by pressing and holding down the <CTRL> 
button while double cUcking additional objects. To magnify the object, the object is 
selected as described above.and the icon 341 is cUcked. 
20 To move the selected objects, the software provides an object navigation icon 353. 

When the icon 353 is selected, object navigational tools appear on the screen 330. These 
navigational tools 355 are shown in Figure 68 and Figure 69. The object navigational tools 
355 comprise a large circle 357, a small circle 359 and a small rectangle 361. First, the 
object is selected as described above. Then, the object navigation icon 353 is clicked, 
25 activating the tools 355 such that they are displayed. The user then positions the mouse 
pointer relative to the tools 355 and presses and drags as described below to position the 
object. When the mouse pointer is outside the large curcle 357, when they start dragging 
the object is turned either clockwise or counterclockwise depending on the direction of 
draggmg. When the mouse pointer is positioned within the large circle 357, when they start 
30 draggmg they rotate the object in any direction. When they start dragging from mside the 
small circle 359, the object is moved in one plane. When they start dragging firom inside 
the rectangle 361, by dragging down the object is moved closer, by dragging upward the 
object is moved farther away. 
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Measuring ohientR 

Referring again to Figure 67. the icon 351 aUows the user to estabUsh a 
measurement marker on any portion of the virtual model of the dentition. Hxe user uses the 
icon 351 to place markers at any two points on the dentition and the distance between the 
markers is displayed on the screen. 

To use the icon 351, the user chcks on the icon, and then cUcks anywhere in the 3-D 
view of the dentition to place markers. A straight line is drawn between two markers. IHe 
distance bet^veen the markers appear, on the screen, e.g.. at the upper left hand comer of the 
wmdowpane of the 3-D view. A tool in the Tools- menu in includes a DELETC ALL 
MARKERS jfunction to delete the markers. 

The measurement function aUows the user to measure tooth size, inter-molar width, 
mter-camne width, the arch length, curve of spee. and other important orthodontic and 
diagnostic parameters. The measurement aspect of the invention is particularly significant 
m that It permits precise quantification of simulated tooth movement, both in terms of 
estabhshing initial treatment plan as weU as monitoring treatment. 

Viewing rr oss-ser.tinns of mode] 

The viewing options also include a clipping plane feature by which cross-sections of 
the teeth through any desired plane are possible. As shown in Figure 73. the clipping plane 
xs shown through the.upper jaw. but the user at can move this plane in three-dimensional 
space at will. If the teeth are magnified, this clipping plane feature is very usefiil for 
inspecting contact pomts between the upper and lower jaw, viewing and adjusting the upper 
and lower jaws in the initial bite registration, and adjusting the location of the occlusal 
plane. For example, in Figure 74 the clipping plane is shown through the upper and lower 
incisors 3 12A and 3 12B. 

The clipping plane is manipulated like an object with the object navigational tools 
shown m Figure 68. Tho plane is accessed using a tools menu and the user highKghts or 
selects SHOW CLIPPING PLANE. A plane appears on the screen. The user then cUcks 
on the object navigation icon 353 (Figure 67). The object navigational controls 355 of 
Figure 68 then are displayed. The user then positions the mouse pointer over the 
navigational controls 353 to adjust the position of the cUpping plane. When they start 
draggmg in the region outside the large circle 357 (Figure 68, Figure 73), the plane is turned 
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Clockwise or counterclockwise. Then they start dragging inside the large circle 357, the 
plane is rotated in the direction indicated by the dragging. When they start dragging from 
inside the small circle 359, the cUpping plane is moved in the direction of ttie dragging. 
When they start from inside the rectangle 361, if they drag up they cut less into the model, 
5 by dragging down they cut further into the model. 

Viewing and adjusting initial bit e registration 

The first step in typical treatment plamiing is deciding where to place the teeth m 
three-dimensional space. This will ordinarily involve a definition or fixation of the 
10 vertical level of the teeth relative to the bones, and defining an occlusal plane, and 
adjusting the occlusal plane sagittally and transversely. This, in turn, will ordinarily 
involves an assessment of structural relationship between the teeth and the maxilla and 
mandible. The orthodontist perfonns this by accessing and studying X-ray, CAT scan, 
photographs or other two dimensional data stored in the patient records portion of the 
15 treatment planning software, and of course the three-dimensional model of the 
malocclusion, with the teeth either represented as individual tooth objects or in context 
wiHi the surrounding anatomical tissue. The mid-sagittal profile of the incisors and 
molars is set up by superimposing the mid-sagittal plane of the teeth over the X-ray. 

Figure 66 is a screen shot from the workstation running the treatment plaiuung 
20 software showing a patient information screen. The screen includes a region 500 for 
storing various photographs 502 of the patient's head and fece, and various views of the 
patients dentition. The photographs are taken with a digital camera and loaded into the 
woricstation, and accessed from the treatment planning software. The patient information 
section of the software also includes separate screens for entering and displaying otiier 
25 pertinent information for the patient, accessed through a menu 504. These additional 
screens (not shown) include the patient demographics, patient medical, dental and 
orthodontic history, examination notes, and X-rays. 

To assist this process, the treatinent planning software provides the ability to view 
and adjust tiie initial bite registration. The initial bite registration between the upper and 
30 lower arches can be modified using the U/L relation tab 327 of Figure 69. The user can 
move or rotate the lower jaw relative to the upper jaw by entering values in the field 366. 
The user can also simulate a chewing motion of the upper and lower jaws by movmg the 
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sUde bar 368 down. During this simulation the lower jaw moves from side to side and up 
and down to simulate a chewing motion. 

Figure 71 shows how the three-dimensional viewing area on the screen on the 
workstation can be broken up into separate screens using known windowpane techniques. 
Here, in windowpane 360, the area of the molar in the observed stage is displayed, with the 
orthodontist able to assess the upper and lower relation and change values for the upper and 
lower relation in three planes of space. Simultaneously, ^vindowpane 362 shows the upper 
and lower jaws as seen from above. Windowpane 364 shows the dentition looking from 
the molars out towards the incisors, a view the orthodontist would otherwise not be able to 
access without the three-dimensional virtual model. These various views, plus the cUpping 
plane tool, and the X-Ray and patient photograph data in the patient records portion of the 
software, provide a complete suite of tools for effective orthodontic diagnosis, treatment 
planning, and appliance.design, including initial bite registration. 

Figure 72 is another screen shot showing the three-dimensional model of the 
dentition in the observed stage. No values have been entered in the malocclusion field 366 
in the U/L Relations tab. By inspection of the upper and lower jaws (using magnification 
or chpping plane features if needed), the user can set numerical values in field 366 and 
mnnediately simulate the corresponding tooth movement to anive at a desired upper and 
lower initial bite registration. 

Design nf Tarf>f» t Archfom^ 

Referring to Figure 75, after the user has set the initial level of the occlusal plane and 
mspected the initial observed situation, the next step is selection of a desired or target 
archform for the patient and the midline. The orthodontist will have previously selected or 
mdicated a general type of archform for the patient, e.g. Roth. The treatment plannmg 
software allows the user to generate a target arch based on this archfomi for the individual 
patient. The user higWights the observed stage in region 356 and then right cHcks the 
mouse. The pop-up menu 370 appears and the user selects INSERT TARGET STAGE. 
The target stage is arrived at by creating a flat virtual wire that nms through the virtual 
bracket slots to create a virtual arch Ime. The arch shape is based on the user's selected 
preference for the arch shape, based on the patient's facial anatomy, bone structure, 
malocclusion, and other parameters using the orthodontist's judgment. The wire target 
shape has a best fit through the malocclusion bmcket slot positions. In one embodiment 
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this wire is flat. . It is also possible to design the wire to adapt to the malocclusion in the 
vertical direction to create a Curve of Spee "if desired. The geometry in the transverse 
direction can also be changed, such as the transverse curve of Monson estabUshing an 
inclination of the teeth in the coronal plane. 

5 Figure 76 is a screen shot from an orthodontic workstation showing the computer 

model of the patient's teeth objects 312 positioned in a target or desired condition. The 
illustration is the result of the user selecting an archform for the patient from a known type 
of archform and the computer placing the virtual brackets along the arch selected by the 
user. This is executed by placing or threadmg the virhial brackets along the archform or 

10 curve selected by the orthodontist. The brackets are omitted from Figure 76, but are shown 
in Figure 77. When the initial target archfomi is created, the sUde Une tab shown in Figure 
67 is activated. 

The initial target archform presented to the user in Figure 76 is only an mitial 
archform. The treatment planning software allows the orthodontist to change many variables 

15 in the target situation, simply by entering new values in the sUde Hne area 402 of the screen 
display. Figure 19 shows some of the parameters by which the orthodontist can adjust the 
shape of the arch, including the distance between the cuspids, the distance between the rear- 
most molars, the center line offset, and the radius of curvature at the front of the teeth. 
SUde hne area also permits the user to select a symmetrical archform or an asymmetncal 

20 archform. iid apply conrections on the right and left quadrants as indicated. As values are 
entered in area 402, the shape of the archform is instantaneously modified on the screen 
display, allowmg the user to simulate various potential archform configurations for the 
patient. 

. " m generating the archforms shown in Figures 76 and 77, the user will ordinarily set 
. 25 up-the lower arch first. . The upper arch is automatically derived from the lower arch. The 
: user'can.view the Mch forms in several ways usmg three arch icons 390, 392 and 394. The 

arch icon 390 is- for the malocclusion arch, which causes a blue line to appear on the screen 
which exhibits the curvature of the observed arch or malocclusion. The line passes through 
" tlie slote on the Virtual brackets, as placed on die teeth. The arch icon 392 is for the target 
30 arch, which represents a custom archwire passing through the bracket slots of the teeth m 
the target situation. The Une 396 in Figure 76 represents this arch. The arch icon 394 if 
for an ideal spline or curve generated by the software to represent an optimal shape. The 
arch retains a parabolic shape as the user manipulates the target arch usmg the entries m the 
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sUde line. The numerical values in region 398 of the sUde line tab represent checlq.oints 
and boundary conditions on the ideal spline associated with the icon 394. These values can 
be edited as indicated. 

Since the software allows for generation and display of a malocclusion archfonn and 
a planned aichfonn, the difference between the t^vo archfonns indicates the space needed to 
control arch length inadequacy; i.e., to identify the need for inteiproximal reduction, 
extraction, or control of gap size. Interproximal reduction can be achieved by the cUpping 
plane feature, and the simulation of shapmg of individual tooth objects in three planes of 
space. The simulation of extractions and control of gap is provided for as explained next. 

Space Manai^emi^t - ManaaemeTit ofV.^rti.r.. o»h Qam R.hve.. T..fU 
The treatment plamnhg software provides a space management tab 344 that is used 
, for managing space between: teeth, including tooth extraction. Figure 82 is a screen shot 
showing the target situation of both the upper and lower jaws. The user clicks on the space 
management tab 344. and the region 420 of the tab appears. The region 420 allows the user 
to smiulate the extraction of a tooth m either the current or target stage. Here the user is 
snnulating the extraction of tooth number 41 by clicking on the appropriate ceU for that 
tooth m the rows for cuireat and target stages. The result is shown in Figure 83. The 
extraction of tooth 41 is simulated. The region 422 also allows the user to enter values 
for the mesial gap size between teeth. A defeult value of 0.1 mm is provided. However, the 
user can change these values. For example, the user can enter any values in the row of cells 
for mesial gap size as shown m Figure 84 (in this example 2mm). Note also in Figure 84 
that the simulation of the extraction of tooth 41 is not being performed since the cell 423 is 
not checked. The tooth moves distally if a positive number is typed in the cell for that 
tooth in the mesial g^ size row 424. If a negative number is typed into the cell, the tooth 
moves mesially. 



Adjusting virhi al brarVet p r>«:iti»T, 

After the upper and lower archfomis have been optimized, the user may wish to 
adjust the position of the virtual brackets 400 on the teeth. The step of adjusting the virtual 
bracket position can also be performed prior to the design of the aix:hform. 

The vertical position of the virtual brackets relative to the incisai edge of the virtual 
teeth is one adjustment that can be made. This distance can be measured using the 
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measurement, tool described earlier. Bracket manufacturers have recommendations for this 
distance. If the initial landmarking has placed the brackets outside of the recommended 
distance, this distance can be adjusted by using the object navigational tools. Alternatively, 
the user can select preferred values for the archform and the bracket set (bracket 

5 manufacture, bracket technique and wire type, e.g., straight) and the landmarks and virtual 
brackets will be placed on the teeth at the recommended distance. 

The bracket placement can also be performed interactively by the user. The user 
looks at every tooth 312 one by one (using a screen such as the screen shot of Figure 75) to 
see if they are basically satisfied with the bracket 400 position, e.g., angulation, side to side 

1 0 position, and its relation to teeth in the opposing jaw. The user may be performing this step 
while viewing the malocclusion, or the target stage. To improve the position of the virtual 
bracket 400, the user selects the bracket, zooms in if necessary, and adjusts the position of 
the bracket on the tooth sxirface using the navigational tools as described earlier. 

15 Correction of Individual Tooth Position In Target Archform 

After the archform has been designed and the bracket placement optimized, the user 
can adjust the individual tooth position on a tooth by tooth basis ui the target arch. There 
are three ways this can be performed. First, the user can use the tables ia the Target 
Correction tab. See, for example. Figure 78, in which the user has entered a value of - 15 
20 degrees for rotation of tooth number 41, and the tooth is rotated by that amount. The 
correction is reaUzed by a bend in the archwire 396, The bracket position on the tooth does 
not change in this example. The target corrections tab 410 permits any values to be entered 
for mesial, buccal and coronal translation m three planes of space, and torque, angulation 
and rotation movements about three orthogonal axis. Thus, independent tooth position . 
25 corrections are available in 6 degrees of freedom for every tooth, merely by entering values 
in the tables in the target corrections tab 410. Another example is shown in Figure 81, in 
which a new target position is shown for tooth number 41 . 

Secondly, the user can move teeth using the Bonding Corrections tab 346 of Figure 
69. An example is shown ui Figure 85. The bonding corrections tab 346 allow the user to 
30 enter new values for tooth position for any tooth in the arch by merely entering values in the 
appropriate ceU. The selected tooth 312A (tooth number 41) is moved as indicated (here, a 
new rotation value of -15 degrees is entered). The virtual bracket remains in the same 
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location in space and the gap 412 between the pad of the bracket and the surfece of the 

tooth is taken up by a bonding adhesive when the bracket is bonded to the tooth. 

Examples of common tooth movements that can be simulated using the bonding 

corrections tab or the target corrections tab are moving the lower incisors buccaUy or 

lingually. inclining the incisors axially, and leveling the incisors. 

Thirdly, the user can simulate tooth position corrections interactively using the 
navigational tools. The user displays a target stage as needed. A tooth object is selected as 
explained above. The user cUcks on the zoom icon 341 of Figure 67 to zoom in or out as 
needed. The user then clicks on the object navigation icon 353 to display the object 
navigation controls. The user then uses the navigation controls to move the tooth as desired. 
The movement of the tooth is recorded as new values in the bonding correction and target 
correction tables, in case the user wants to quantify the movement or use those tables for 
further modification of the tooth position. After the user has moved the tooth to the new 
position, they click one of two check mark icons 414, 416 (Figure 77, 78) that are 
higMighted on the screen. The blue check mark 414 realized the new tooth position via a 
bonding correction. The red check mark 416 reaUzes the new tooth position via a wire 
correction. 

Another example of coixection of individual tooth position is shown in Figures 79 
and 80. hi Figure 80, the target situation is shown, both with the virtual tooth objects 312 
and the virtual brackets 400. Note that with tooth 16 (312A), there is a gap between the 
rearmost cusp of the tooth and the opposing tooth. The orthodontist can correct this gap by 
building in a bracket offeet, basically repositioning the location of the bracket 400 on the 
tooth 312A by entering an amount in the table 450 in the angulation ceU for tooth number 
16 (here - 15 degrees). The result is shown in Figure 80. The change in the angulation value 
causes tooth number 16 to rotate back mto a more desirable occlusion with the opposing 
tooth. 



Wire Tab 



The wire tab 432 of Figure 86 allows the user to make wire configuration changes, 
while not changing bracket position. Note that in Figure 86, the virtual wire 396 is shown 
isolated from the teeth and brackets. The virtual wire can be scaled to actual size and 
printed out and provide a template for a manual bending of a wire if the orthodontist 
chooses not to obtain a wire from the precision appliance manufacturing center. The tab 
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includes a section 430 where the user can view the distance in mm ia which the wire will 
slide relative -to the brackets, on a tooth by tooth basis, when the teeth are moved from the 
current situation.to the target situation. Note that a collision is detected for tooth 22 in 
movement pf .thejooth from the current situation to the target situation This can be 
5 resolved in several possible ways in accordance with the teachings of U.S. Patent 
^ [ to Sachdeva et ?l1., serial no. 09/451,609 allowed December 7, 2000, the 

contents of which are iacorporated by reference herein. 

, Additional WireBending Corrections 

10 The wire offsets tab 426 (see Figure 73 and Figure 87) allows the user to simulate 

bending a wire 396 to the corrected tooth position while simultaneously retaining the 
original; bracket position. Note that in Figure 87 the wire 396 is flat with no corrections 
indicated. ^ The user highlights one of the teeth in the virtual model and enters new values 
for tooth position. The change is reflected in a new shape for the wire 396, The tab also 

15 allows the user to build in over/under compensation into the archwire. These settings do 
not affect the bracket position. The characteristics of the wire bends, such as the radius of 
curvature, can be controlled by accessing a slidings tab (not shown). Another tab, forces 
428, displays approximations of the forces the wire appUes to each tooth to displace it from 
its initial position to the target position. 

20 

Treatment Stages 

Since the difference between the current situation and the target situation is 
quantifiable in terms of millimeters of movement or degrees of rotation about three axes, the 
treatment for the patient can be broken up into segments or stages with each stage defined 

25 arbitrarily. For example, the orthodontist can specify a first treatment stage as Hlo 
movement of the teeth from the initial position half-way to the final position. The software 
includes a screen that permits the user to view the position of the teeth when moved to the 
half-way position. Basically, the simulation simply multiplies the differences in tooth 
position between initial and target stages by 0.5 and the resultmg tooth positions are 

30 displayed 'on the workstation. Additionally, the user can specify further intermediate 
positions,, such as one fourth or three fourths. With this feature, the orthodontist can 
monitor treatment and compare the progress of the treatment with the limits or stages that 
have been set. When the patient comes in for a visit during treatment, the patient's 
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dentition is scanned. The three-dimensional model of the now cuirent situation is compared 
with the defined stages and perhaps with the target situation. Difference between actual 
progress and the planned- treatment can be quantified Changes to the archwire can be 
designed using the treatment plamnng software to move the teeth in the desired direction to 
account for unexpected biologicdl influaices that may be observed. 

The above description of treatment planning has been predicated on the use of a 
three-dimensional model of the dentition from the scanner, obtained as described above. 
However, it is possible to perform digital treatment plamiing by importing into the software 
three-dimensional software from other sources. Is it known today that three-dimensional 
models can be exchanged with^ different software programs in different file formats, similar 
to the translation programs that convert text documents from one type of file (e.g., 
Microsoft Word to WordPerfect). Most three-dimensional appUcations have several import 
filters for different 3D formats. 

However, there are generally two different ways to describe three-dimensional 
objects: by surface representations and by solid representations. A 3D file that holds 3D 
data in a surface description consists typicaUy of triangles that form the surface of the 
object. The STL format is one of the oldest and therefore most common formats that uses 
triangles. It is used to feed information to stereoHthography machines. A more detailed 
description of STL can be found at http:y(/www.mnisonline.com/artciles.019704.htail, the 
contents of which are incorporated by reference herem. 

Treatment Monitoringr 

Interactive, computer-based treatment monitoring is a significant advantage provided 
the freatment planning and appHance design aspects of the system described herein. 
Typically, when the patient comes into to the office during treatment, they will be scanned 
and a new digital model of the dentition is acquired. From this new model, differences can 
be monitored between the current situation and the original malocclusion, and differences 
between the current situation and the target situation or pre-defined Umits or treatment 
stages as defined earher. These differences can be quantified with precision. For example, 
a point on the tooth in the cuirent model is selected, and the model of the tooth at the 
original malocclusion is overlaid on the screen. The superposition of the two teeth aUows 
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the user to view the change in position that has occimred. The measurement marker features 
described earher aUow the user to quantify precisely the amount of movement. 

Any deviations between the therapeutic result that is observed and the expected 
result can be captured precisely and at an early stage in treatment using the scamiing and 

5 treatment planning features described herein, and corrected for. For example, the 
orthodontist may need to place additional bends in the archwire. Such additional bends can 
be perforaied by simulating the wire shape on the screen, displaying the wire only on the 
screen, and printing out the screen and using it as a template for bending the wire. The 
current situation could also be forwarded to the precision appUance center for manufacture 

10 of a new appliance. Of course, these monitoring and treatment corrections are appHcable to 
any type of appliance selected for the patient 



20 



Part 5.' Appliance Manufacturing 
15 The appUance that is manufactured in accordance with the treatment plarmed for the 

patient can vary within the scope of the invention and will depend on the application, 
including the type of appUance desired by the orthodontist and the patient. Obviously, the 
treatment planning and simulation features described above can be used in wide variety of 
treatment regimes, including flat wires and brackets, finishing wires, retainers, Herbst 
devices, expansion devices, and removable, transparent aUgning devices such as those 
furnished by AUgn Technologies. For example, the movement of the teeth firom the current 
or observed stage to the target or treatment stage can be broken into a series of movement 
steps. For each step in the process, the position of the teeth in that stage is known by virtue 
of the manipulation of the individual tooth models in the treatment planning software. 
25 These tooth positions can be used to manufacture the aUgning devices. 

In a representative embodiment, the results of the treatment planning software are 
used to generate a customized orthodontic archwire and a bracket placement tray for precise 
placement of off-the-shelf brackets onto the patient's teeth. When the treatment planning 
features have been executed to the satisfection of the orthodontist and the proposed target 
30 situation finaUzed, the treatment planning software will store the following mformation (m 
addition to the patient records): 

1) the virtual model of the current stage or malocclusion; 
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2) the placement location of the brackets on the malocclusion, including the type and 
dimensions of the brackets; 

3) the orthodontist's selection of a type of archwire (including material and size); and 

4) the target situation, including the location of the teeth and brackets in three dimensions 
at the target situation. 

Note that it is not absolutely necessaiy for the apphance manufacturing step to 
calculate or even know the shape of the archwire. Archwire geometry is dictated by bracket 
positions in three-dimensional space when the teeth are in the target situation. This bracket 
position information is included in the target situation, no. 4) above. 

With reference again to Figure I, the above infomiation from the treatment planning 
software is sent over a suitable communications medium 24 in digital form to the precision 
apphance service center 26. The service center manufactures a customized archwire and a 
bracket placement tray for placement of the brackets at the intended location on the teeth in 
the malocclusion. 

Basically, the position of the bracket slots, and the shape of the brackets, when the 
teeth are in a target situation, is information that is ultimately developed and stored by the 
treatment planning software. This position of the bracket slots and the shape of the slot 
(e.g., the length) is of course known in three dimensions. From the slot shape, it is possible 
to derive a three-dimensional set of line segments that represent the shape of an archwire 
passing through the bracket slots in the target situation, and calculating the optimal shape of 
bends that comiect the bracket slots together. The positions of the straight sections and Hie 
bends are fed as an input ffle to a wire bending robot. The wire bending robot need only 
know the wire size, the sh^e and size of the slots, and the positions of the slots in the target 
situation. From this infomiation, robot commands are generated to bend the archwire into 
the desired shape. 

Hie bracket placement tray is separately manufactured using stereolithography or 
other similar technique. The treatment planning software generates items 1) and 2) above, 
and superimposes the brackets on the teeth to generate a three-dimensional model 
compnsmg the three-dimensional tooth objects plus the virtual brackets at their intended 
locations in the observed stage or malocclusion. This three-dimensional model is suppUed 
to a stereoUthography (SLA) instrument The SLA instrument manufactures a plastic model 
of the teeth with the brackets superimposed on the teeth. A thermoplastic foil is placed 
above the SLA model and the model and foU are placed ^vithin a pressure chamber. The 
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chamber is pressurized so that the foil envelops the dentition and the brackets. After 
cooling, the foil is removed j&om the model.. The foil, now in the shape of a transfer tray, 
has small indentations where the brackets are located. Real brackets are placed in these 
indentations. The orthodontist uses indirect bonding techniques to bond the brackets to the 
5 teeth. The transfer tray positions the brackets on the teeth at the desired locatioa After the 
bonding is complete, the orthodontist removes the transfer tray, resulting in the brackets 
bonded to the teeth at the desired location. A further scan of the dentition can be made at 
this step to verify the position of the brackets. Any substantial deviation in the bracket 
position can be accounted for by modification of the archwire, again using the treatment 
10 planning features described above. 

There will always be some small gap between the bracket base (the part bonded to 
the toothy and the tooth, as an off-the-shelf bracket will never precisely match the individual 
tooth of any given patient. One option is to fill the gap using a surplus of bonding adhesive 
during bonding. Another option is to equip the base of the bracket with a customized pad 
15 made fi'om an adhesive. 

Customized bracket pads can be manufactured using a variety of techniques. One 
possibility is to bond a blob of adhesive to a bracket base, and mill the blob using a milling 
machine to match the tooth surface. Since the tooth surface is known precisely firom the 
scanning, and the position of the bracket base relative to the surface is also known precisely, 
20 a very accurate bracket pad can be manufactured in this fashion. Another technique is to 
stamp out a bracket pad using stamping machine either in the desired shape or as a blank 
and milling the stamped pad to the desired shape. A third possibility is creating a negative 
of the customized pad, forming tibie pad in a mold, bonding the pad to the bracket and the 
orthodontist trimming the pad as necessary when the pad is bonded to the tooth. 
25 Once the brackets have been bonded to the teeth, a scan of the bracket placement is 

made. The scan is compared to the digital template of the expected bracket position. If the 
bracket is placed incorrectly, the bracket can be re-bonded to the tooth. Alternatively, 
corrections to the wire may be made if necessary to account for displacement of the brackets 
from their expected position. BasicaUy, this is done by simulating the position of the teeth 
30 with the actual bracket placement at the target situation and correcting the shape of the 
archwire as necessary, or obtaining a new archwire based on the actual placement of the 
brackets. 
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It is also possible to manufacture a transfer tray without the intermediate step of a 
positive SLA model representing the dentition and the brackets. These trays may be either 
smnlar to the plastic sheets and just be fabricated using rapid prototyping methods (SLA, 
laser smtering, milling, 3-D printing, etc.), or they have more sophisticated geometry 
includmg features to hold the brackets and features to provide proper positioning at die 
dentition. AdditionaUy, bracket placement jigs can be miUed from the three-dimensional 
models of the dentition and the brackets using techniques described in, for example, in the 
Andreiko patent cited earlier. 

As another possibiUty, the customized bracket bonding pads based on the virtual 
tooth^racket geometry can be febricated. Such bonding pads may include additional 
material which confonns to the surface of flie tooth to act as a placement jig for the tooth to 
aUow the bracket bonding pad to be precisely located on the tooth. THen the bracket is 
bonded to the bracket bonding pad. The additional material is then removed. 

With the above description in mind, it will be appreciated that the treatment 
planmng functions can be performed with the importation of any 3D object from any source. 
While the preferred embodiment uses the scamied three-dimensional model from the hand- 
held seamier, this is not necessary or required. The software preferably has hnport filter 
for common types of files of 3D objects including STL, DXF, VRML etc. 

Additionally, another key aspect of the treatment planmng software is that "it permits 
the placmg of brackets onto tooth models in an arbitrary mamier and virtuaUy "bonding" 
those brackets to the teeth so tiiat they move together. While the present embodiment has 
descnbed a landmarking feature by which the user places landmarks on the teeth and the 
brackets are placed on the landmarks, this may be perfomied in other ways, including 
automatically with no user involvement based on parameters such as crown height, bracket 
type, tooth size, etc. 

As indicated above, the software also provides for aHgning of the brackets along 
vutual wu-e of any shape. Tlius, as the user changes the shape of the archwire, the brackets 
follow this shape, thereby indicating tooth position correction in the target situation. This 
feature allows the user to drive the treatment plamiing based on wire shape. Conversely, the 
wn:e shape can be dictated by the placement of the brackets on the teeth. Thus, the 
treatment plan can be one driven by bracket placement. Obviously, wide variety is possible 
in the shape and design of the wire. 
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Furthermore, by providing for the simulation of teeth in both maxiUa and mandible 
together, the software provides for a wide variety in defining the maxillary/mandible 
relationship, the occlusal plane and the mid-line of the teeth. 

Another significant aspect of the software is that it provides for virtually unlimited 
5 3D measuring features using the marking icon and measurement feature described earlier. 
This feature offers a powerful diagnostic tool, as well as a tool for monitoring the progress 
of treatment and quantifying results. 

Because the teeth are represented as complete three-dimensional virtual objects, it is 
possible to detect the colHsion between teeth or between teeth and brackets in the simulation 
10 of movement of teeth from the current to the target situation. The point cloud representing 
tooth objects defines a surface, when surfeces come in contact during the tooth movement a 
collision is simulated. This colUsion is detected by suitable collision detection algorithms. 
When the collisions are detected, the user can be notified of the colUsion and resolve the 
conflict between teeth, for example by selecting one tooth as a primary tooth and movmg 
15 that tooth first in an initial stage of treatment, and then moving the other tooth in a 
secondary stage of treatment. These features are described in fiorther detail in US Patent 
appUcation serial no. 09/451.609 filed November 30, 1999, now allowed, the contents of 
which are incorporated by reference herein. 

Another advantage of the instant treatment planning software is that it offer the user 
20 real time shnulation with immediate feedback on the effect of user specified tooth 
movement. When a value is entered into a field in the display or the user uses the 
navigation tools, results are displayed immediately. Further the system offers arbitrary 
access to every object, including election, navigation and export. 

25 A. Robot design 

Figure 88 is a schematic representation of a presently preferred archwnre 
manufacturing system 34 shown in Figure 1. The key aspects of the system 34 are shown in 
block diagram form in Figure 8SA. The system 34 includes a control system for 
controlling the operation of a wire bending robot 604. The control system in the illustrated 
30 embodiment comprises a general purpose computer 600 running a bending program and a 
robot controller 602. The computer 600 receives an input file firom the precision appliance 
service center computer that contains information as to the location of bracket slots m three 
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dimensions, when the teeth are in a target situation. The computer 600 suppUes the robot 
controller 602 with wire position infonnation coiresponding to points along the wire where 
bends need to be made. This infonnation is translated by the controUer 602 into motion 
commands for the wire bending robot 604. 

It wiU be appreciated that the system works in an analogous fashion when bending 
other types of medical devices. The computer 600 receives an input file from some source 
that provides information as to how the medical device in question needs to be bent. The 
computer 600 suppHes the robot controUer 602 with position infonnation coiresponding to 
points along the length of the medical device where bends need to be made, and the robot 
responsively bends a medical device in accordance with the input file. 

The wire bending robot 604 consists of a moveable arm 606 having a gripping tool 
at the distal end thereof The moveable aim has a proximal end mounted to a table or base 
610. The robot also includes a first gripping tool 608. In the iUustrated embodiment, flie 
first gripping tool 608 is fixed with respect to the base 610 and mounted to the table. 
Hence, the first gripper tool 608 will be refenred to herem occasionally as the "fixed 
gtipper." It would be possible to place the first gripper tool 608 at the end of second robot 
ann, m which case the first gripper tool would not necessarily be fixed, but rather would be 
firee to move in space relative to the source of archwires, or the other moveable aim. In 
such as system, a coordinate system would be defined having an origin at which the first 
tool is positioned at a known position. The bending commands for the robot would be with 
respect to this known point. 

A wire or other workpiece to be bent is held by the first gripper 608 and the gripping 
tool at the end of the moveable ann 606, and the arm is moved to a new position in space to 
thereby bend the worfcpiece. The details of the constmction of the ann 606 and fixed 
gripper 608 are described m fiirther detail below. 

The system 34 of Figure 88 is set up to manufacture customized aichwhres one after 
the other continuously, as would be case in a precision appliance service center serving a 
plurahty of clinics. As such, the robot 604 includes a source of arcfawire material. The 
source could be a spool of wire m which case a cutting tool is needed to cut lengths of wire 
for individual aichwires. Alternatively, as shown m Figure 88 and 93 the source consists of 
a magazine 614 containing a plurality of straight archwires 615 ready to be grasped by the 
gripping tool at the end of the moveable ami 606. In an embodiment in which the first 
gripping tool is mounted to the end of a moveable ann, the first gripping tool could retrieve 
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the next workpiece while the moveable aim places the finished workpiece at an exit 
location. 

After ah archwire is bent in accordance with an input file supplied to the computer 
600, the moveable gripping tool at the end of the robot arm 606 places the wire (or 

5 workpiece being bent) at an exit location indicated at 616. A conveyor system 618 
including a plurality of trays 620 is provided for carrying the fiinished archwires wires 622 
from the exit location 616 to a labeling and packaging station 624. The labeling and 
packaging station 624 includes a printer 626 that prints a label for the archwire and a 
magazine 628 containing a supply of packages such as boxes or bags for the wires. A 

10 worker at the station 624 takes a label firom the printer and applies it to the archwire 622 or 
to the package for the wire. The conveyor system 618 is also based on a commercially 
available, off-the-shelf conveyor system, such as of the type available from the Montech 
division of Montrac. 

The wire manufacturing system 34 includes a heating controller 612 responsive to 
15 commands and settings from the wire manufacturing computer 600. The controller 612 
controls the supply of current to heating elements 607A and 607B in the gripping fingers m 
the gripping tools in the robot, to thereby heat the gripping fingers above ambient 
temperature. Temperature sensors 605Aand 605B detect the temperature of the gripper 
fingers and are used for feedback control of the heating of the gripper fingers. A direct or 
20 indirect system for measuring the temperature of the workpiece may also be provided, such 
as infrared heat detector. The heating controller 612 also controls a wire heating power 
supply 611 that siipplies a current to the gripping, tools when they are bending a wire. The 
power supply 611 is used when the robot is bending shape memory materials or Titanium 
Molybdeiium Alloys (TMA) materials, or possibly other materials. The current produces a 
25 resistive heating in the wire. The current is controlled via a wire heating cuxreat sensor 6 13 
so as to produce a wife temperature at which a bend formed in the wire is set into the 
material. The heating of the gripping fingers avoids excessive heat loss when resistive 
heating of the wire is performed. 

Figure 89 is a perspective view of a moveable robot arm 606 used in the 
30 manufacturing system in Figure 88. In Figure 89, the gripping tool at the distal end 634 of 
the arm is omitted, hi a preferred embodiment, the moveable arm is based on an off-the- 
shelf six-axis robot arm and fixed tool. A suitable arm, fixed tool, and robot controller for 
the robot 604 is available from StaubU Unimation of Germany. The Staubli robot arm and 
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fixed tool is customized with gripping fingers, heating controller and ancillary systems, 
software, current heating subsystem., force sensors, and other features as described herein 
for bending archwires or other suitable medical devices. 

The arm 606 consists of a proximal end or base 630 which mounts to the table 610 
of Figure 88 and a free distal end 632 consisting of a tool flange 634, where the second 
gnppmg tool 651A of Figure 92is mounted, as described below. The arm 606 is capable of 
motion along six different rotational axes, with the directions indicated by the arrows 
numbered 1 - 6 in Figure 89. Thus, the base is fixed with respect to the table 610 and the 
head portion 636 rotates relative to the base 630. A joint hi head portion 636 rotates arm 
segment 638 about an axis indicated by arrow 2. Similarly, the arm segment 640 is rotated 
about an axis indicated by arrow 3 by a joint 639. A joint 642 rotates an arm segment 640 
about the axis indicated by the arrow 4. A joint 644 attached to the end of arm segment 640 
rotates flie tool flange 634 about the axis indicated by arrow 5. A sixth joint (not shown) 
rotates the tool flange 634 about an axis indicated by arrow 6. 

Figure 90 is perspective view of the robot arm of Figure 89. showing the movement 
of the arm joint 639 and the corresponding motion of the arm segment 640 and tool flange 
634. The motion commands for the robot are supplied from the robot controller 602 along a 
cable 650 which plugs into the base 630 of the robot 

Figure 91 is a detailed perspective view of a preferred gripping tool 651A that is 
mounted to the tool flange 634 at the distal end of the robot arm 606 of Figure 89 and 
Figure 90. The construction shown in Figure 91 also appUes to the fixed gripper 608 of 
Figure 88 and Figure 92. Hie gripping tool 651A consists of a pair of opposed gripping 
fingers 652 and 654 which open and close to grip and release the workpiece, in this case an 
orthodontic archwire 615. A pneumatic cylinder actuates gripping finger 652 by moving it 
about an axis relative to finger 654, to thereby permit the fingers 652 and 654 to grip and 
release a woricpiece such as a wire. A positioning sensor 655 (Fig. 88A) detects the position 
of the fingers. 

In a representative embodiment, the archwres are made ftom a ^e memory aUoy 
sucl> as Nitinol. a material based on Nidcel and Titanium plus Copper or other alloy. These 
mater^ls can be heated to retain the shape of a bend fonned in the wire. Accordingly, the 
wire heatmg power supply 611 of Figure 8SA supplies a current to the gripping fingers of 
the feed gripping tool and the moveable gripping tool. The flow of current along the wire 
creates a resistive heating of the wire sufficient for the material to take a set according to the 
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shape of the wire as it is bent. To avoid dissipation of heat from the wire into the gripping 
fingers, the gripping fingers 652 and 654 are preferably heated by electrical heating 
elements 607A and 607B. Conductors 660 and 662 supply current to the heating elements 
in the gripper fingers. 

5 The gripping tool 651 A of Figure 91 ftirther includes a force sensor 664 in the form 

of a.strain gauge. The force sensor is designed to detect forces that the wire imparts to the 
gripping fingers after a bend has been placed m the wire or during the bending movement. 
The forces detected by the force sensor 664 are determined both in magnitude and m 
direction in three-dimensions. The use of the output &om the force sensors to overbend 
10 wire is explained in further detail below. 

Figure 92 is a perspective view of the fixed gripper 608 having a gripper tool 651B 
as shown in Figure 91 along with a moveable gripping tool 651A.located at the distal end 
632 of the moveable arm 606. An orthodontic archwire 615 is shown gripped by the 
gripping tools 651A and 65 IB. The fixed gripper 608 and gripping tool 651B remam 
15 stationary relative to the base 610. The archwire is bent or twisted by the fixed grippmg 
tool 651 grasping the archwke 615 and the moveable gripping tool 65 lA also grasping the 
archwire 615, and then moveable gripping toolmg 651 A bendmg the wire by movmg to a 
new location in three-dimensional space relative to the fixed gripping tools. The location m 
space for the moveable arm to move to is determmed by the ixtpnt file fed to the robot 
20 computer 600. Basically, the mput file consists of a series of point locations in a three 
dimensional coordinate system which conrespond to bracket locations and orientation m a 
three-dhnensional coordinate system for the arch, as described m more detail below. The 
manner of calculation of these points and generating movement commands (i.e., arm 
position and switch states for the gripper fingers) for the robot's moveable aim and 
25 commands for the fixed gripper to bend the wire will be described in fiirther detail below. 

Other possibilities exist for input files and calculation of the bending points. For 
example, in extraction cases, the wire is needed to close a gap between teeth and the wnre 
serves as a guide or rail for the bracket to slide along to close the teeth, hi this situation, a 
smooth curve is needed between the teeth to allow the brackets to sUde the required amount. 
30 In this situation, the space between the teeth is divided into small sections, and wure 
coordmates are obtained for each section. A series of small bends are formed at each 
section to generate the requked smooth curve. It may be helpful m this situation to round 
the edges of the gripping fingers to help provide the desired smooth shape. As anotha: 
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alternative, free-fomi curves can be fonned by bending die we between tvvo points which 
would encompass a plurality of brackets. 

While the preferred embodiment of a robot ami is shown in Figure 89, that is not the 
only possible arrangement of a robot ann. The robot of Figure 89 is optimized for complex 
bends and twists in archwires. However, some medical devices or archwires may need only 
simple bends, m which case a lesser number of joints may be required. For example, a one, 
two or three axis robot may be sufficient for some appKcations. 

Figure 95 is a perspective view of an alternative arrangement of a six-axis moveable 
robot ann. hi this embodiment, the robot arm comprises a base 700, a motor that moves 
aim segment 702 along direction 704, a second section that contains a motor that moves 
second aim segment 706 along direction 708, and a third section 710 that contains a motor 
movmg the arm section 712 along direction 714. A motor is provided for rotating aim 
section 712 about axis a. A section 716 is connected to the end of section 712 and includes 
a motor for rotation of section 716 about an axis indicated by p. A sixth section 718 is 
rotated about an axis mdicated by j. The gripping tool 65 1 A is momited to the end of the 
section 718. Robots of this type are also known and suitable for forming the basis of an 
orthodontic archwire bending robot. The term "moveable arm" as used in the claims is 
intended to interpreted broadly to encompass the aim segments of the type shovm m Figure 
95 as well as the construction shown in Figure 90. 

The gripping fingers of the gripping tools 651A and 652 preferably opthiiized, in 
terms of their physical configuration, for the type and shape of the workpiece bemg bent 
This shape may change depending on the nature of the workpiece, e.g., wire, fixation plate, 
spectacle fiames, etc. In the case of wires, wires come in various cross-sections and sizes. 
It may be deshrable to form a pluraUty of contours m the grippmg fingers so as to enable the 
gripping fingers to grip several different types and sizes of wires without changing gripping 
tools. For example, one part of the gripper fingers has a series of rectangular contours to 
grip wires of rectangular cross-section of varying sizes, and perhaps one or more circular 
contours to grip round wires. 

The force sensors on the grippmg tools may also be used to provide feedback for an 
adjustable grippmg force to be appUed to the woricpiece (e.g., wires). It may be desirable to 
allow the wire to slide through the gripper fingers if the forces acting from the workpiece to 
the gripper exceed a certain limit When these forces are sensed, the fixed gripper loosens 
its grip on the workpiece and allows it to slide. 
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Figure 93 is a perspective view of a maga2itie 614 of Figure 88 that holds a plurahty 
of straight archwires needing to be bent in an presently preferred embodiment. Figure 94 is 
a detailed perspective view of the moveable gripping tool grasping one of the archwires 
from the magazine of Figure 93. 

5 The magazine 614 consists of a tray 670 having a set of parallel raised elements 672 

•that define a series of grooves 674 in the upper surface thereof The archwires 615 are 
placed m the grooves 674. The archwires are maintained spaced apart from each other in 
the tray. This permits the robot's moveable gripping tool 651 A to pick up a single archwire 
at a time from the magazine 614 as shown in Figure 94 and insert it into the fixed gripping 

10 tool to commence a bending operation. Also, the magazine 614 is positioned at a known 
location and the dhnensions of the tray and slot features thereof are known precisely. This 
location.infonnation is supplied to the robot control software and allows the gripping tool 
651A to remove the archwires one at a time from the magazine automatically and without 
human assistance. When the magazine 614 is empty a foil one is placed at the same 

15 location. 

Figures 96A and 96B are perspective views of alternative magazine constructions to 
the magazine of Figure 7. In Figure 96A the magazme 614 consists of a cylindrical holder 
with a plurality of apertures 680 spaced from each otiier, each containing a straight archwure 
615. In Figure 96B, the archwires are in a rectangular holder with the apertures arranged m 
20 rows and columns. In either case, the moveable aim grips an individual one of the 
archwires and removes it firam the magazine by virtue of the spacing of the wires fix>m each 
other in the magazine and because the location of each wire m the magazine can be known. 

Figure 97 is a schematic iUusti^tion of a conveyor system 722 including a conveyor 
belt 724 that carries archwires 615 to the robot. Here, the robot is enclosed within a safety 
25 cage 726. A source feeds archwires 615 into slots 728 in tiie conveyor belt. When a wire 
has been bent and the robot is ready for a new wire, tiie belt advances one position and the 
robot grips the next wire placed on the belt 724. As another alternative, a spool of 
archwire can be fed to the robot and a cutting tool (not shown) provided for cutting the wn:e 
from the spool into a desired length. The cutting tool could be incorporated into tiie end of 
30 a second robot am, for example. Still fiorther implementations are possible. 

It also possible for the archwire manufacturing system to have other workstations or 
workplaces in which one or more of the following tasks may be performed: loop bending. 
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surface refming, and marking of the wires. These stations could be positioned at locations 
around the conveyor system 722 or be in separate locations. 

It is also possible to enclosed the robotic wire bending system within an enclosure 
and fill the enclosure with an inert gas such as nitrogen. The inert gas prevents oxidation of 
the workpiece during bending or oxidation or other chemical reaction affecting the gripping 
tools. 
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10 B. Archwire Manufacture 

The production flow for manufacturing archwires (or other similar appliances) with 
a representative embodiment of the wire manufacturing system of Figure 88 is shown in 
Figure 98. The production flow includes the step 800 of loading a wire magazine 614, 
such as spool of wire in an alternative embodunent, feeding the we to the robot at step 802 
and cutting the wire to length at step 804. At step 806, a series of bends are placed in the 
archwire in accordance Avith the prescription for the archwire. After the bending is 
complete, the wires are labeled at the station 624 at step 808 and packaged in a box or other 
package at step 810. 

The bending of the wire at step 806 is based on slot data for bracket slots at 
described below in conjunction with Figures 99-106, or based on some other suitable 
criteria as explained herein. Hie wire bendmg computer 600 receives this slot data from 
the precision appUance center computer of Figure 1. The computer 600 executes a bending 
program that processes the slot data into a set of points in three dimensional space and 
calculates movements of the moveable arm necessary to achieve the appropriate bends in 
flie wire. The computer 600 has a software interface 812 to the robot controller, which 
translates position or movement signals for the robot ann into low level instructions for the 
robot controUer 602. The robot controUer executes a robot control program (ad^ted from 
the control program that comes with the lobot) which causes the robot ann 606 to move 
relative to the fixed gripper 608 to bend and/or twist flie wire. Where the archwire is a 
shape memory alloy, the wire heating power supply 611 supplies current to the gripper 
fingers 652 and 652 on the moveable arm and the gripper fingers on the fixed gripper 608 to 
heat the wire while the wire is held in the bent condition, and/or during bending motion, to 
set the shape of the wire. 
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RnhotTnPutFile 

The input file, which dictates the shape of an archwire after bending, will now be 
discussed in conjunction with Figures 99-106. The input file includes a set of matrices, one 
5 matrix for each bracket in the arch of Ihe patient Each matrix consists of a combination of 
a vector of location of a point on the bracket and a matrix of orientation, indicating the 
orientation of the bracket in three-dimensional space. Both the vector of location and the 
matrix of orientation are based on the position of the brackets on the teeth when the teeth are 
in a target situation. The target situation is developed by the orthodontist firom the scan of 
10 the dentition and the execution of a treatment plaiming using the treatment planmng 
software at the clinic. 

Figure 99 illustrates the- target situation for one arch 820 a patient. The target 
situation is a three dimensional virtual model of the teeth 822 in which virtual brackets 824 
are placed, for example, on the labial surface of the teeth. A coordinate system is defined 
15 for the arch 820 having an origin 826. The coordinate system is in three dimensions, with 
the X and Y dimensions lying in the plane of the arch and the Z direction pointing out of the 
page. The location of the origin 826 is not particularly important. In the illustrated 
embodiment, an average "mass" is assigned to each virtual tooth in the arch, and a center of 
"mass" is calculated for the arch 820 and the original 826 is located at that center. 
20 As shown in Figures 100 and 101, a vector of location 828 is defined for each 

bracket. The vector 828 extends firom the origin 826 to the center of the slot 830 in the 
bracket along the wall 832 of the bracket slot, i.e., to point 834. The vector of location 
consists of the X, Y and Z coordinates of the point 834 in the defined arch coordinate 
system, 

25 The orientation matrix consists of a 3X3 matrix of xmit vectors of the form: 







Yi 


Zi 
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X3 


Y, 


Z, 




X3 


Y3 


Z3 




where X^ 


X. 



30 Figure 101, Y, Y, and Y3 are the X, Y and Z components of the Y unit vector shown in 
Figure 101, and Z, Z^ Z3 are the X, Y and Z components of the Z imit vector shown in 
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Figure 101. As noted above, the matrix for each bracket thus consists of the combination 
of the 3 X 3 orientation matrix and the position matrix, and is thus as follows: 



X, Y, Zi X 
X Y2 Zj Y 
2) . X3 Y3 Z3 Z 

0 0 0 1 

where X, Y and Z in the right hand column of entries is the position vector. 

The robot input file also includes an antitangential value and a tangential value for 
each bracket. The antitangential value consists of the distance from the center of the bracket 
slot (point 834) to a point defining the temiinus of the previous bend in the wire. The 
tangential value consists of the distance from the center of the bracket slot to the point 
defining the terminus of fee next bend in the wire. The input file also consists of the 

thickness of the wire, as measured in the direction oftheY unit vector in Figure 101. 

With reference to Figure 102, an example of the 4 X 4 matrix 2) for a rear most 
molar of a patient will be described. Figure shows the origin 826, the position vector 828, 

and the X and Y unit vectors which indicate the orientation of the bracket slot. Figure 102 
also shows the scale (in units of millimeters) which gives absolute location and orientation 
information for the bracket slot. Here, we assume in tiae example that there is no Z 
component to tiie tangential vector X or the normal vector Y Figure 103 shows the 
tangential distance T, and tiie antitangential distance AT, as measured along the centerline 
of the archwire. The resulting mabix is shown in Figure 104. 

From a set of the matrices as shown in Figure 104 comprising aU the brackets in tiie 
arch, the robot bending program extiacts a series of line segments in tiiree dimensional 
space, which are defined by the terminus of the antitangential and tangential distances for 
each bracket slot. The set of line segments 840 is shown in Figure 105. THe line segments 
are defined by a set of points PI, P2. P3 . . . Pn having known tiiree dimensional 
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coordinates due to the known, location of the bracket slots and the known tangential and 
antitangential distances. The line segments can also be defined as a set of vectors havmg 
a location for tiie head of the vector and a magnitude and orientation in three directions. 
The folloAving discussion will use the set of points PI, P2, P3 . . . FN. In Figure 105, the 

5 slashes 842 indicate the end points of the bracket slot 830 of Figure 101 . 

The bends need to be placed in the wire before point PI, between points P2 and P3, 
between points P4 and P5, etc., that is, between the bracket slots. The slot-to-slot bends of 
the complete archwire are bent section by section. To fonn one slot-to-slot bend, the wire is 
fed so that the fixed gripper tool 65 IB and the robot arm gripper tool 65 1 A can grip the wire 
10 in its initial shape. The wire length between fixed gripper and robot arm gripper is equal to 
the curved length of the wire along the bend. The straight wire sections 840 between the 
bends have to fit to the bracket slots. To bend the wire into the demanded shape, the mam 
control computer 600 sends signals to the robot controller 602. The robot controller 602 
generates signals to move the robot arm 606 with the robot gripper tool 651 A into a new 

15 position. The movement path is defined by a bending trajectory. The bend is indicated at 
844 in Figure 106. 

To form one slot-to-slot bend (e.g., bend 844 between P2 and P3), there might be 
several of these bending movements necessary. One slot-to-slot bend is considered finished 
if two consecutive straight wire sections (e.g.. between PI and P2 and between P3 and P4), 
20 have the desired relative positions between one another. 

To achieve this position, there are different approaches dependent on the wure 
material properties possible: a) bending material with elastic/plastic properties, such as 
stainless steel, b) bending material with shape memory properties, and c) bending TMA 
alloys. 

25 Material with elastic/plastic properties must be overbent to. compensate for the 

elastic part of the deformation. The overbend process, which is described in further detail 
below, can be defined as a closed loop contiol. Within the first bending step, the robot arm 
606 moves to a new position. Preferably the new position is equal to the planned position 
or to the planned position plus an amount of overbending. At tiie end of the move the forces 

30 and moments acting on the grippers are measured. They indicate tiie remaining elastic 
deformation in the wire. To detemune tiie .gripper position which correspond to the released 
wire shape, the robot arm 606 starts a new move in direction opposite to the acting forces 
and moments. The forces correspond to a translational move, tiie moments to a rotational 
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move. By adjusting continuously the movement direction to the measured forces and 
moments, the robot achieves a position, where the forces and moments are in the order of 
the measurement resolution (zero-force-position). By choosing an appropriate measurement 
resolution, the remaining elastic deformation can be neglected and the relative position of 
the two grippers corresponds to the relative position of the straight we sections in the 
released situation. This zero-force-position is compared to the plamied position. If the 
differences are bigger than the tolerance limits, an additional bending step follows to 
decrease the difference. From the zero-force-position the robot moves now in direction to 
the planned position and overrides the plamied position about the value of the difference 
between zero-force and plamied position. The endpoint of this move is called oveibend 
position. From the overbend position starts again the force and moment controUed move to 
find the new zero-force-position. If the new zero-forx:e-position is within tolerance limits to 
the plamied position, then the bending process for one slot-to-slot bend is completed and the 
wire is fed to bend the next slot-to-slot section. If the amount of overbend was too much, the 
new overbend position is calculated as described above. If the amount of overbend was not 
sufficient, then the new overbend position is calculated as the former overbend position plus 
the difference between new zero-force-position and plamied position. The described 
process is repeated within a loop up to the situation, that the difference between zero-force- 
position and planned position is smaller than the tolerance limit. 
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Materials with shape memory properties and TMA wiU be bent to the planned 
position. To transfer this position into the memory of the alloy, the wke section between the 
two grippers is heated to a certain temperamre for a certain time. The heating is possible e.g. 
by conductive resistance heating, laser, convection, radiation, or applying warn air or liquid 

5 to the material. Heating current and time must be appropriately adjusted to the respective 
alloy, the wire section length and the wire shape. To wami-up the wire, the wire heating can 
start akeady during the bending movement to the planned position. To avoid a heat sink 
effect at the gripper fingers and to ensure that the complete inter-bracket section of the wire 
obtains the necessary heating, the gripper fingers 652, 654 (Figure 91) or at least the contact 

10 areas of gripper and wire are heated too. The grippers may be heated continuously during 
the production process of the whole archwire. To compensate for an incomplete transition of 
the bending position to the alloy memory, there can be defined a certain amount of 
overbending. 

In bending TMA materials, the material can be heated to a high temperature where 
15 there is no springback, however when the material cools, it retains its springback properties. 
The procedure for bending such materials is as foUows: 1) heat the gripper fingers; 2) bend 
the wire to tHe desired configuration; 3) heat the wure up to the temperature where the 
springback tendency no longer exists; 4) turn off the heat source and allow the wire to cool, 
and 5) advance the wire to the next position for bending; and then repeat steps 1) -5). 
20 The bending of tiie wire firom one section to the next requires that the location and 

aHgmnent of one straight wire section (i), for example P3 to P4. is defined in reference to 
the previous straight wire section (i^l) in the counting order defined in the bending system. 
The origm of the bending system is defined at the end of the straight wire section 
which aims towards the following straight section (i). The x-axis is equal to the direction of 
25 the straight wire section (i-1) directed to section (i). For wires with rectangular cross-section 
the y-axis is perpendicular to x and in direction to the wider dunension of the wire. For 
quadratic or circular cross-section the y-axis must be perpendicular to x and can be chosen 
according to practical reasons. The x,y,z-axis follow the right hand rule. 

The bracket slot data as described above needs to be transformed to bending data for 
30 use by the robot controller 602. This is done by calculation, firom the position of the 
bracket center point 834 (Figure 100) to the position of the straight wire section center point 
(located along the middle of the wire, point 840 in Figure 105). 
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Next, there needs to be a calculation of the bent wire shape and length. In Figure 30, 
this is shown as the shape of the wire between points P2 and P3. For each bending step, the 
robot gripper grips the wire in a certain distance from the fixed gripper corresponding to the 
length of the wire between points P2 and P3. THe wire section between the two grippers will 
be bent. To nnninnze the bending forces and moments and to ensure that the bending forces 
and moments don't exceed limits, which may cause damage to the equipment or to the wire, 
the gapped W length should be approximately the "natural" length of the wire in its bent 
shape. If the length is too short, the wire will be torn and there will be high tensional forces, 
if It's too long, the wire will tend to kink. 

The robot computer 600 therefore calculates the approximate shape of the bent wire 
usmg an appropriate algorithm. One way is deriving a second or third order curve 
representing the shape of the wire using numerical techniques. Another would be using a 
regular spline algorithm. Ideally, there should be no sharp bends in the wire. In the 
lUustrated embodiment, a Bezier spline algorithm is used. n.e algorithm gives an 
analytical description of a smooth curve and generates a set of points along the length of the 
cm^e. me length of the curve is obtained by summing up the distance (in tee 
dmiensions) along each segment of the curve. The separation distance between each point 
m the segments can be set arbitrarily and in the illustrated embodiment is 0.05 mm. The 
algorithm is as follows: 



Input: 

. Centerpoint location and aligmnent of two neighbored straight wire sections (given 
as 4x4matrice with local vector /, tangential vector ? nonnal vector n and vertical 
vector V) These matrices correspond to the 4 x 4 matrix described earUer. 

• tangential and antitangential distances J, and 

• Bezier-distance (empirical value) 

The Bezier formula, as known by Uterature, is described by four points as shown in Figure 
20A. The points of the sphne curve are given by: 

•:^+(i-v)'.v.:^+(i-v).v'.^+v'.:^ .^{o^D 
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Here it will be noted that the Bezier points to P4 in Figure 20A are not necessarily the 
points P.1-P4 of the wire segments shown in Figure 19, but rather are the points used to 
calculate the Bezier spline length and shape. 
5 To describe the curved wire shape between the straight wire sections from slot 

(i - 1) to slot i , the Bezier points Pj , ^ ^3 ^ -^4 are calculated by: 
10 P^^l-s,,yt, 

The wire length L and the N intermediate spline points can be calculated by the algorithm 
shown m Figure 20B. 

The empirical value Bezier-distance must be set to make the calculated and 

15 actual bent wire shape tally. For orthodontic wires, a good assmnption is 6^ = Ix . . . 2x the 

larger wire cross-section dimension. 

The bending trajectory needs to be calculated for each bend. The bending trajectory 
is a number of positions of the moveable arm's gripper 651A in relation to the fixed gripper 
65 IB, which connect the start position and the destination position of a bending movement. 
20 In general there are translational and rotational movement components from each bending 
trajectory position to the next. 

F'or each bending trajectory position the calculated length of the curved wire must be 
equal to ttie calculated length in the planned position. To avoid kinking condition for the 
wire die movement can be divided into two parts: 
25 r. - Initial movement, to steer the wire iuto a distinctly deformed shape. The initial 

movement is defibaed as a rotational transfomation around an axis through the 
endpoint of the fixed gripper pCTpeadicular to the connecting vector of the start 
position and the end position. This is indicated in Figure 115A by the rotational 
motion of the moveable gripping tool 65 IB and movements a, b, c, and d. 
30 2. Finish movement to destination position. The finish movement is a gradual 
approach from the start position (or if there is an initial movement from the end 
position of the initial movement) to the destination position. The translational and 
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rotational parts of the whole bending movement are split up steadily to the 
individual bending trajectory positions. Between two bending trajectory positions, 
the movement path of the robot gripper is defined by a linear translational movement 
along the straight line connection of the two positions and by steadHy divided 
rotational movement. The distance between txvo bending trajectory positions must be 
small enough to avoid kinking of the wire and inadmissible forces and moments. 
These movements are indicated at positions d, e, :^ and g in Figure 1 15B. 
In bending wire as described herein, the robot system 604 of Figure 88 has a 
coordinate system in which an origin S60 is defined as shown in Figures 109 and 110. The 
W passes through the fixed gripping tool 651B through a small gap formed when the 
fingers are in an open position. The origin 860 is defined as the along the center of the axis 
of the wire at the planar edge of the fixed gripping tool 651B. The rabot controller 
software knows the location of the moveable gripping tool relative to the fixed gripping tool 
in this coordinate system at aU times. Furthermore, the wire is gripped by the moveable 
gripping tool at a precise point on the moveable gripping tool. Therefore, when the wire is 
held by the fixed gripping tool and the moveable gripping tool, the distance between the two 
is known exactly. This aUows the bending shown m Figure 115A-115C to be executed 
without stretching or contracting the wire. In particular, the distance as measured along the 
wire between the fixed and moveable gripping tools at the point of attachment to the wire is 
constantly maintained a distance equal to the calculated Bezier distance for the wire as bent 
between the points P2 and P3 of Figmre 105 and 106, and of course for subsequent bends. 

To advance the wire between bends or to place the wire in condition for the first 
bend, there are at least two possibiUties. One is that the moveable gripper tool grips the 
wire and pulls it through the fixed gripping tool (with the fixed gripping tool opened to 
allow the sliding of the wire with respect to the gripping tool). As an alternative, the wire 
could be on a spool or coil, and the spool rotated by a motor to advance the wire through the 
fixed gripping tool, hi the later embodiment, a cutting tool will need to be provided to cut 
tile wire after the bending is completed. Archwire manufachirers sell wires in bulk already 
cut to length and tiie present description is made in the embodiment in which the wire 
segment is advanced by the moveable gripping tool advancing the wire through tiie fixed 
gripping tool. 

Having the bent ware between the two grippers in a tensed state, the robot gripper is 
moved to a new position, where no forces and moments are acting on tiie gripper. The force 
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sensors 640 on the fixed and moveable gripping tools are used to detennine ±s position. 
This position is called the zero force position and corresponds to the released state of the 
wire. Forces, moments and the movements components are calculated in the main robot 
coordinate system of Figures 109 and 110. 
5 Depending on the nature of the material, some overbending of Ihe wire may be 

needed. This would be indicated for example if the zero force position is not the same as 
the calculated position for the robot's moveable aim 606. To better understand the 
overbending principles, attention is directed to Figures 108 and 108. 

Figure 107 illustrates the relationship between force and deflection of wire. The 
10 solid line 866 iadicates how much force is needed to give a certain deflection of the wire. 
Where the force is less than Fl, when the force is released the whre returns to its initial state 
and experiences no deflection due to elastic properties of the wure. At force level Fl, some 
permanent deformation, i.e., plastic deformation, of the wire starts to take place. With a 
force level of F2, the wire is deflected an amount D2, but when ttie force is release from the 
15 wire the wire bends back to a deflection amount Dl, with the relaxation curve 868 
essentially parallel to the curve 866 up to force level Fl, as shown. Thus, some level of 
force indicated at F3 is required to be appUed to the wire such that when the force is 
removed the required deflection Ds is achieved in the wire. The fact that F3 is greater than 
F2 and that the deflection D3 is greater than D2 illustrates that some overbending is 
20 genially needed to yield the proper shape of the wire after the bending forces are removed. 

Figure 108 illustrates the stresses mvolved with wire material whra. it is bent. The 
wire has one portion experiencing elongation stress, as indicated at 856, while compression 
stresses are experienced at locations 858, with fee length of the arrows indicating the 
relative magnitude of the stress. With small bends, the forces at the center of the wire are 
25 small enough such that only elastic deformation occurs, while at the outside portion of the 
wire some plastic deformation may occur. The result is that bending wire is a mixture of 
elastic and plastic deformation, the character of which is determined by the amount of the 
bend, the wire material, and the cross-sectional shape. 

To determine the amount of required overbending, there are several possibilities. 
30 One is a purely analytical solution hke finite element analysis of wire. Alternatively, a 
piece of wire can be tested to determine its response to known forces, and the result stored 
as a calibration table of bends. BasicaUy, the curves in Figure 107 are obtamed 
experimentally. A more prefrared approach uses force sensors 640 (Figure 88A) on the 
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fixed and moveable gripping tools to sense the zero force position of the wire and compare 
the location of the moveable gripper in this position with the intended position. A 
geometrical or deformation approach is another alternative. In this approach, the wire is 
bent some amount and then released, the relaxation position noted, the wire bent some 
more, the wire released, etc. the process continuing until the wire is bent to the desired 
position. 

With a force based system, perhaps augmented by an adaptive, self-learning artificial 
intelligence type learning program or cahbration table based on previous bends of similar 
wire, the resulting configuration of the wire can usually be achieved more quickly. 
BasicaUy, for every bend performed in the wire, information is stored as to the movement 
necessary to result in a specific bend. For example, to achieve a 13 degree bend in the wire 
of type T and cross-sectional shape W, the wire had to be bent 15.5 degrees, and this 
information is stored. With enough data, a mathematical relationship can be derived that 
that rq,resents curves 866 and 868 for the wire of type T (at least in the portion of the curve 
of interest), and this mathematical relationship can be used, in conjunction with force 
sensors, to quickly and accurately place the required bends in the wire. 

In either situation, an optical system such as a camera could be used for detecting flie 
position of the whre in the relaxed position is used to determine the actual shape of the wire 
after any given bend. 

Figure 111 is a flow chart of a deformation controlled overbending process that can 
be used with stainless steel or other wires, including shape memory wires where some 
overbending is still required. 

At st^ 870, a calculation is made of overbending values in both a translation and 
rotational aspect. This calculation could be performed for example using finite elements 
methods, using a cahbration table, using a derived mathematical relationship between force 
and bending, using stored values for overbending from previous bends, or some 
combination of the above. 

At step 872. the bending curve is calculated iq, to the planned position and 
including the overbending values. TTiis involves the Bezier spline algoridmx set forth 
30 previously. 

At step 874. the moveable gripping tool is moved to the position indicated by the 
sum of the planned position plus the overbending position. Tins forms a bend in the wire. 
Agam, this position is detemiined in reference to the robot coordinate system and in 
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reference to ifie spatial relationship between tiie points where a bend needs to be placed in 
the wire (P3 and P2 in Figure 105, for example). 

At step 876, the force sensors are used to measure the residual forces imparted by 
the wire onto- the gripping tools, and if the forces are greater than some threshold, the 
5 moveable gripping tool 651 A is moved to the position where the force measurement is zero 

or substantially zero. , - 

At step 878 the actual position of the moveable gripping tool is measure using the 

position sensors; in the moveable robot arm. 

At step 880, a check is made to see if the difference between the actual position and 

10 the planned position is less than a limit. If not, new overbending values are calculated (step 
882), aiid a new bending curve is calculated to overbend the wire an additional amount, in 
the desired direction, to bring the wire closer to flie desired shape (step 883). 

St^ps 874-883 are repeated until the difference between the actual position of tiie 
moveable gripping tool and the planned position is less than a limit. , 

15 At step 884, the error in the actual position relative to the planned position is noted 

and compensated for by coirecting the next slot position. In particular, the next slot position 
represented by the next pair of points in the set of points in Figure 105 is translated in three 
dimensions by the amount of the error. This correction of the next slot position is needed so 
as to avoid propagation of an error in any given bend to all subsequent bends. 

20 At step 886, the overbending results from steps 874-882 are saved in memory and 

used in an adaptive technique for estimating future overbending requirements as explained 
above. 

Figure 112 is a schematic representation of the overbending performed by the 
method of Figure lir. The line 900 represents the slot between the fingers of the fixed 
25 gripper and the line 902 represents the wire in various stages of bending. The dashed line 
904 indicates the movement of step 874, with line 906 representing the slot of the moveable 
gripping tool between the gripping fingers where the wire is gripped in the planned position. 
The zero force position where the moveable gripper is moved to is indicated at 908 (step 
876 in Figure 111). There is both a rotational (dw) and translational (dx) aspect to tiie 
30 difference between tiie zero force position and the planned position. The last overbend 
position is shown as position 910 (the overbend calculated at step 870). The required 
correction is shown by the position 912. This new overbend position, calculated at step 880 
in Figure 1 i 1 is equal to the last overbend position 910 plus the planned position 906 minus 
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the zero position 908. This new overbend position includes both translational and 
rotational aspects as indicated by -dx and -dw. 

One possible example of actual robot gripper movements to feed the wire through 
the giippers and execute a representative bend will be explained conjunction with Figure 
113A-113E. As shown in Figure 113 the wire is thread through the fixed gripper tool 
651B or placed there by the moveable gripping tool such that some length of wire is 
extending beyond the end of the fixed gripping tool 651B. The points PI and P2 along the 
wire segment are indicated. The moveable gripper tool 651A is positioned above and 
sUghtly to the right of the fixed gripping tool, here 0.2 mm away. The moveable gripping 
fingers open and the moveable gripper moves down to clamp the wire. The 0.2 mm 
distance is merely chosen so that the fingers to not colhde and can vaiy &om the iUustrated 
embodiment. Tlie fingers cooperate with each other by translation movements of the 
moveable gripping tool and releasing and closing the fixed and moveable grippers such that 
the wire is advanced through the fixed gripping tool. This is also indicated by Figure 1 13B. 
showing the fixed gripping tool 651B open (OP) to aUow the wire to be shd through a slot 
formed between the fingers of the tool 651B. The moveable gripping tool moves to the 
right to draw the wire through the fixed gripping tool until the point P2 is to the right of the 
fixed gripping tool (as shown in Figure 113C), where it can be grasped by the moveable 
grippmg tool. As shown in Figure 113C and 113D. the moveable gripping tool opens and 
releases its grip on the wire (indicated by OP) and moves to the position where it closes 
(CL) and grasps the wire at location P2. Then moveable gripping tool 651A draws the wire 
through the fixed gripping tool while gripping the wire at point P2, such that point P3 is 
located at the origin of the robot coordmate system, as shown in Figure 1 13D. Since the 
planned location of both P2 and P3 after a bend in the wire is made is known in the robot 
coordinate system, the moveable gripping tool 651A moves to the position shown in figure 
113B to place a bend in the wire. At this point, if fiirther overbending is caUed for, the 
process of, e.g.. Figure 1 II and 112 is performed to place the required overbend in the wire. 
The movements of Figure 113B-113D could be combmed to one movement if the distance 
is small enou^ and dependmg on the thickness of the wire. 

Figure 118 illustrates before and after positions of the wire when the bending of 
Figure 113E occurs. The figure illustrates that the movement of Figure 113E is not a 
straight Une. movement which might cause excessive elongation or kinking of the wire. 
Rather, the movement of the gripper is illustrated as steps a, b, c. d, e, f such that the 
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distance, as measured along the length of the wire, is maintained constant. The movement 
may performed in two stages (such as shown in Figures 115 A and 1 15B). The result is that 
two bends 912 are placed in the wire, as shown in Figures 115C. Of course, a twist could 
also be performed in the wire if required by the prescription. 
5 After the bend has been placed in the wire, the steps shown in Figure 116A-116D 

are performed to advance the wire along to the position of the next bend. First, as indicated 
at Figure 116A, the moveable gripping tool 651 A is translated to the right an amount 
indicated by AX. This moves point P3 to the right of the fixed gripping tool by an amount 
AX, as shown in Figure 1 16B. Then, the moveable gripping tool releases the wire and re- 
10 grips the wire to the right of the fixed gripping tool as shown in Figure 116C and agam 
translates to the right an amount sufficient to move point P4 to the right of the fixed 
grippmg tool. The moveable gripping tool releases the wire again and grips the wire at 
point P4. The wire is again translated to the right such that the situation shown in Figure 
116D is obtained. The wire is now in position for a bend in the wire between P4 and P5. 
15 The process of Figure 111 and 112 occurs again to calculate new bending and overbending 
positions and the bend is formed in the wire. The process of Figure 1 16A-1 16D continues 
until all the bends have been formed in the archwire. When the final bend is complete, the 
wire is released from the moveable gripper at the exit location of the whe manufacturing 
system, and carriedTjy conveyor to the labeling and packaging station described earher. 
20 Shape memory alloy materials require heating to take on the shape given by the bend 

produced in Figure 1 1 3E. Thus, for these wires, while the wire is held in the position shown 
by Figure 113E, heat is appUed to the wire to raise the temperature of wire to the value 
needed to take the set. The temperahare varies with the type of material. In the illustrated 
embodiment, a resistance heating system is used as described previously. The current is 
25 adjusted until the proper temperature is reached. The heat treating is deemed complete 
when the force sensors read zero (or less than some Umit) when the wire is held by the 
grippers m the planned position. The amount of current and time applied to the wu:e is 
again stored information that can be used for future heating of the same type of wire. 

For some softer shape memory materials, e.g., NiTi, the force sensor 640 (Figure 
30 S8A) provided in the gripping tools must be very sensitive to detect the small forces 
involved. While shape memory materials may not requure force sensors at all, they can be 
used to give information as to the effectiveness of the heating process. 
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In a preferred embodiiaent, two force sensors are used A coarser force sensor, used 
for measuring larger forces during bending, is fitted to the moveable gripping tool. A finer 
force sensor, with a hi^er resolution, low noise and higher sensitivity, e.g., with a 
sensitivity of less than 0.0005N, is fitted to the fixed gripping tool, in order to detect the 
5 zero force position. The force sensors are both based on strain gauge technology and can be 
readily adapted firom commercially available strain gauges or off the shelf units. For 
example, the finer force sensor may have different ampUfiers or other modifications to the 
circuitry to have greater sensitivity, signal to noise ratio and force resolution. Other types of 
force sensors, such as those based on piezo technology, would be suitable. Suitable off- 

10 the-shelf strain gauge force sensors are available firom IR3 Inc. of Woodland California, 
model nos. 45E15A-U760 (fixed gripping tool) and 67M25A-I40 (moveable gripping tool). 

Other types of heating systems could be adopted for archwires and other types of 
woiiq)ieces to .be bent, such as laser, flame, infiared, conductive or radiant heating. Some 
s^ringback may still be observed in shape memory materials even when heating is 

15 performed unless the wire is heated close to the maximum permitted temperature of the 
wire. Therefore, with some shape memory materials it may be desirable to perform some 
overbending in order to lower the temperature needed to set the new shape into the wire. 
Again, the required amount of overbending at a given wire temperature can be stored in 
memory and used to derive a relationship between temperature, overbending and resulting 

20 position for the material, which can be used for subsequent heads in the wire. 

Due to the complexities of wire deformation and twisting in wire that can occur 
when, wn-e of a rectangular cross section is bent, and the difficulty in controlling the 
resulting shape of the wire (particularly when complex bends and twists are formed in the 
wire), the usage of force measuring devices, and position sensors to detect the shape of the 

25 wire when the wire is in a zero force condition, gives accurate information as to fiie shape of 
the wire after a bend. Thus, a force based approach to overbending is a preferred 
embodiment. The actual position of the wire in the zero force condition can be obtained by 
position sensors on tiie robot arm (which makes no contribution to the measurement of 
forces), or better yet, by releasing the wne firom the moveable arm and detecting the 

30 position of the wire with a camera or other optical system. Basically, the camera would 
image the wire immediately in front of the fixed gripping tool. Pattern recognition and 
image processing algorithms are used to determine the edge of the wire, and thereby 
calculate its shape. More than one camera could be used if necessary to image the wire 
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sufficiently to calculate twist in the wire. The effects of gravity would have to be 
compensated for in any position measuring system in which the wire is not held by the 
moveable gripping tool. 

Thus, in one possible embodiment the robot further comprises an optical sensor 
5 system such as a CCD camera detecting the shape of the orthodontic appliance after the 
bend in said orthodontic appliance has been made, such as by releasing the appliance from 
the moveable gripping tool and allowing the appliance to take its natural position, and then 
using the optical system to detect the shape of the appliance. 

It is also possible to use both the force measuring systems and the optical system as 
10 a final check on the shape. The force sensor system (e.g., coupled to the fixed and/or 
moveable gripping tools) detects forces generated by the orthodontic appliance after tiie 
orthodontic appliance has been bent. The moveable aim is operated to move the orthodontic 
appliance to a zero-force position in which the forces detected by the force system are below 
a predetermined threshold. The optical sensor system detects the shape of the orthodontic 
15 appHance in the zero-force position. The position detected by the optical sensor system can 
be used as a feedback mechanism for further wire bending if the zero force position is not 
the intended or desired configuration of the appliance. An optional type of sensor system 
would be calipers that contact the workpiece and responsively provide position information 
as to the location (i.e., bend) formed in the workpiece. 
20 For stainless steel wires, there is generally no need for heat treatment of the wire. It 

is simply bent into the desired position, with overbending performed as required. The 
shorter the distance between endpoints of a bend, the greater the deformation in the wire, 
therefore the greater the predictability in the deformation. With orthodontic archwires, the 
situation can occur where there is a relatively long distance between bracket slots 
25 (particularly in the region of the molars) and it can be difficult to obtain a stable bending 
result. A preferred solution here is to make this distance shorter by adding on some length 
to the tangential distance of one slot position and the antitangential distance of the next slot 
position, as shown in Figure 117. Here, point P2 is extended in space to point P2% and 
point P3 is brought closer to point P2 by moving it to point P3\ The required bending of 
30 the wire is now calculated relative to points P3' and P2'. The bend placed between P3* and 
P2' is now sufficiently short that it can be formed with enough control. 

In practice, it known that after an archwire has been fitted to the patient's brackets, 
the archwire imparts forces to move the teeth to the desired position. However, after a 
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certain amount of time, some smaU amount of bend remains in the wire but it is insufficient 
to cause any further tooth movement. Consequently, the teeth are not moved to their 
desired position. This can be compensated for by adding an additional amount of bend to 
the wire so that when the wire is installed, it will continue to exert forces until the teeth have 
been moved all the way to their desired position. As shown in Figure 118, this smaU 
additional bend is shown as 920. Figure 33 shows the wire of Figure 118 installed in the 
brackets of a patient. The bend 920 of Figure 32 is in addition to other bends that may be 
placed between the brackets 824. Figure 1 19 illustrates that enough residual force exists 
by virtue of bend 920 to move the teeth 822 to their desired position. 

In certain orthodontic situations, loops may need to be bent in the wires. Figure 120 
illustrates a loop 922 formed m a wire 615. The loop may be fonned by the robot of Figure 
90. Alternatively, only the peripheral comers 924 of the loop 920 are formed by the 
bending robot, as shown in Figure 121A, with the remainder of the loop formed by placing 
the wire over a die 926 having a shape 928 matching the shape of the bottom portion 930 of 
the loop 920. A forming tool 928 is moved against the wire and die 926 to form the bottom 
portion of the loop as indicated in Figure 121B. 

The robot may also include a separate ami or toohng by which stops, or other 
features are bonded to the wire by suitable techniques such as welding. For example, fee 
robot can be fitted with other tools for fomiing a Herbst appUance or expansion devices. 
Alternatively, different types of wires could be joined together by welding. 

The robot may also be used to bend clear, transparent wires made from polymeric or 
plastic materials, such as thermoplastics, duroplastics, polyacryUc plastics, epoxy plastics, 
themioplastics. fiber reinforced composites, glass fiber containing materialss or other 
similar materials suitable for an orthodontic archwire. These plastics archwires may require 
heating during bending, but current sources may not be suitable heating devices. 
Recommended techniques for heating the plastic wire mclude blowing hot air over the wires 
during bending, using heated pHers, placing a heat conductive material onto, the wire, using 
a laser to heat the wire, or spraying a hot vapor or Uquid onto the wire. 

As noted above, additional possibifities are presented for bending fixation plates, 
orthotic devices, prosthetic devices, endodontic devices, surgical guidewires, surgical 
archbars, implants or surgical tools with the robot manufacturing system. The gripper 
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fingers and associated structures may be optimized depending on the workpiece or 
appliance in question. However, the principles of operation are basically the same. 

For example, the robot of the present invention is particularly useful for bending 
fixation plates, rods, compression plates and the like, for example facial, cranial, spinal, 
5 hand, and long bone and other osteosynthesis plates, such as, for example, the titanium 
appliances provided by Leibinger Gmbh of Gemiany. These fixation plates may consists of, 
for example, an elongate skeletal frame having a plurality of apertures for receiving screws, 
arranged in straight lengths, C, Y, J H, T or other shape configurations, or a long cylindrical 
rod. At the present, these appliances are manually bent by the surgeon to the shape of the 

10 bone in the operating room using special manual bending tools. It is possible to automate 
this process and bend the plates in advance using the principles of the present invention. In 
particular, the shape of the bone or bone firagments is obtained a CAT scan, firom a scan of 
the exposed bone using a hand-held scanner (such as described above. Once a three- 
dimensional virtual model of the bone is obtained, e.g., firom CAT scan data, the virtual 

15 model is manipulated using a computer to fiise the bones together in ttie desired position, 
The surgeon then overlays the three-dimensional virtual implant in the desired location on 
the virtual model, and bends the virtual implant using the user interface of a general purpose 
computer storing the virtual model of the bone and implant. The required shape of the 
implant to fit the bone in the desired location is derived. 

20 Alternatively, a physical model of the bone in the desired configuration can be 

manufactured fi-om the virtual model using stereolithography (SLA), three-dimensional 
lithography, or other known technology, and the shape of the implant derived firom the 
physical model. 

As another alternative, a SLA physical model of the bones (e.g., skull) is made firom 
25 a CT scan or oflier source, and the surgeon performs a simulated surgery on the physical 
model to place the bones in the desired condition. The model is then scanned with an optical 
scanner and a virtual model of the bones in the desured condition is obtained, as described in 
the patent application of Rudger Rubbert et al., cited above. The virtual fixation device is 
then compared or fitted to the virtual model of the bones to arrive at a desired shape of the 
30 fixation device. 

In either situation, the shape of the implant is then translated to the robot controller 
as a series of straight sections and bends of known geometry (and specifically position 
commands for the moveable gripping tool relative to the fixed gripping tool). The 
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moveable and fixed gripping tools of the bending device grip the hnplant or fixation device 
at one end and then either bend the apphance or advance the position of the implant to the 
location of the next bend, and continue along the length of the device to fonn the device in 
the desired configuration. Obviously, some modification to the gripping tools may be 
needed from the disclosed embodiment depending on the physical characteristics of the 
device being bent, and such modifications are within the ability of persons skilled in the art. 

The bending apparatus described above is also adaptable to generic workpieces, such 
as tubes, cylinders, wires or other types of structures. 

The bending apparatus may use resistive heating, force sensors, overbendmg, and 
the other features described at length in the context of orthodontic archwires, dq,ending on 
the application for oflier workpieces. 

Bonding Pads 

Customized bracket pads can be manufactured using a variety of techniques. One 
possibiHty is to bond a blob of adhesive to a bracket base, and mill the blob using a milling 
machine to match the tooth surface. Since the tooth surface is known precisely from the 
scanning, and the position of the bracket base relative to the surface is also known precisely, 
a very accurate bracket pad can be manufactured m this feshion. Another technique is to 
stamp out a bracket pad using stamping machine either m the desired shape or as a blank 
and millmg the stamped pad to the desired shape. A third possibility is creating a negative 
of the customized pad, fomiing the pad in a mold, bondmg the pad to the bracket and the 
orthodontist trimming the pad as necessary when the pad is bonded to the tooth. 

Since the treatment planning software gives precise information as. to the location of 
the bracket relative to the tooth surface in three dimension, the shape of the . customized 
bonding pad can be detennined with high precision. This shape is^ansfeired to a suitable 
milling machine using known techniques and the pad is milled in accordiice with the 
desked shape. The pad is then affixed to the bracket (either at fixe precision apphance center 
or at the orthodontic clinic and the bracket + pad bonded to the tooth. 

Other inventive features disclosed hftrmii 
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Drthn/lnntir Apvliance Ovtimization 

Generally, the present invention also provides for a method and apparatus for arch 
wire receptacle (e.g., brackets, bands, headgear tubes, etc.) optimization. Such a method 
and apparatus includes processing that begins by obtaining a digital model of an orthodontic 
5 structure of an orthodontic patient. This may be done in accordance with the teachings 
described above. The processing then continues by retrieving a digital model of an initial 
arch wire receptacle (e.g., a bracket, a band, a headgear tube, etc.) that was selected ftom a 
pluraHty of digital models of arch wire receptacles. Generation of a digital model of an arch 
wire receptacle may be fiarther performed as described above. The processing then continues 
10 by digitally placing the digital model of the initial arch wire receptacle on a given tooth of 
the digital model of the orthodontic structure to provide a digital axch wire receptacle 
placement. The process then proceeds by retrieving a digital model of an arch wire. For the 
given tooth, the processing continues by digitally modeUng a force system on the toolh 
based on flie digital model of the initial arch wire receptacle, the digital arch wire receptacle 
15 placement, and the digital model of the arch wire. Having done this, when the force system 
is not optimal, the force system is modified to achieve an optimal force system. The force 
system includes the effects of bonding pad thickness, bonding agent properties, the arch 
wire, Ugation forces, and biological characteristics (e.g., growth, tooth interference, tooth 
anatomy). Each of these factors may be adjusted individually or in combination to optimize 
the force system. With such a method and ^aratus, optimal force systems given 
ortiiodontic parameter constraints may be obtained without the limitations of prior art 
orthodontic practices. 
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Determining and Monitoring Orthodontic Treatment 



GeneraUy, the present invention also provides for a method and apparatus for 
treating an orthodontic patient. Such a method and apparatus include processing that begins 
by generating digital information regarding tiie orthodontic patient by a site orthodontic 
system. The digital information may mciude a three-dimensional scan image of the 
30 patient's orthodontic stracture (e.g., teeth, jaw bone, gums, and other facial features), x-rays, 
clinical examination and measurements, dental history, medical history, photographs, etc. 
The site orthodontic system then ti^smits the digital information to an orthodontic server. 
The orthodontic server and/or the site orthodontic system creates an electronic patient 
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record from the digital information. The electronic patient record includes, but is not 
limited to. clinical examination interpretations, radiology examination measurements, 
automatic and manual cephalometric analysis, a created electronic cephalometric tracings, 
an electronic composite including an integration of the three-dimensional images, 
radiogr^hic data (e.g., x-rays, CAT scans. MRI scans), photographs, a generated electronic 
articulation module, measurements and analysis of electronic models, data quaUty assurance 
checks, and/or supplemental data. 

The orthodontic server then generates an initial treatment from the electronic patient 
record, wherein the initial treatment plan includes precise steps to obtain a desired 
orthodontic structure. In other words, the initial treatment plan includes a series of steps, 
wherein, for each step, a corresponding orthodontic apparatus wiU be designed to reshape 
the patient's orthodontic structure into a desired structure for the precise stvp. The 
orthodontic server then transmits a digital version of the initial treatment plan to the site 
orthodontic system. Upon confirmation from the site orthodontic system, the orthodontic 
server designs an orthodontic apparatus (e.g., brackets, bands, wires, head gear, mbber band 
placement, and/or a retainer) for one of the precise steps based on the treatment plan. Tlie 
orthodontic apparatus is then fabricated/assembled and provided to the site orthodontic 
system. In addition, the orthodontic server may provide practitioner and/or patient 
instructions for use, maintenance, and/or mstallation of the orthodontic apparatus. 

Upon receipt of the orthodontic apparafes, the orthodontist verifies receipt of the 
orfliodontic apparatus and subsequently installs it. At a predetennined point in time after 
the instaUation in accordance with the treatment plan, the patient's mouth is electromcaUy 
scanned to obtain updated digital information. The site orthodontic system provides the 
updated , digital information to the orthodontic server, which uses the iqjdated digital 
information to update the electronic patient record. From the updated electronic patient 
record, the orthodontic server determines whether the actual movement of the patient's teeth 
is as predicted. Such a determination may be supplemented with input from a practitioner 
(e.g., a technician, dental assistant, orthodontist, specialist, consultant, etc.) at the site or at a 
remote site wife respect to the patient. If so, the next step of tiie initial treatinent plan is 
executed (e.g., the wire for the next step is fabricated, provided to the orthodontist, and 
installed). If, however, the actual movement is not as predicted, tiie orthodontic server 
adjusts the freatinent plan to obtain the desired results. After the treatinent plan has been 
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adjusted, the next step of the revised treatment plan is executed. This monitoring of a 
patient's progress and revising the treatment plan, when necessary, continues throughout the 
treatment. Thus, with such a method and apparatus, a scientific approach is provided to 
orthodontic treatment throughout the treatment, maintains the treatment costs at reasonable 
levels, and provides a more consistent and reduced treatment time. In addition, the present 
method and apparatus allow orthodontic practitioners to have just-in-time inventory, allows 
for treatment adjustments that would be required due to changes in tooth position and/or jaw 
development, and further aUows for changes in the orthodontic structure if the force system 
becomes sub-optimal. 

Treating an Orthodontic Patient 



Generally, the present invention also provides for a method and apparatus for 
treating an orthodontic patient. Such a method and apparatus include processing that begins 
by generating digital information regarding the orthodontic patient by a site orthodontic 

15 system. The digital mformation may include a three-dimensional scan image of the 
patient's orthodontic structure (e.g., teeth, jaw bone, gums, and other fecial features), x-rays, 
clmical examination and measurements, dental history, medical history, photogr^hs, etc. 
The site orthodontic system then transmits the digital information to an orthodontic server. 
The orthodontic server and/or the site orthodontic system creates an electronic patient 

20 record from the digital information The electronic patient record includes, but is not 
Hmited to, clinical examination interpretations, radiology examination measurements, 
automatic and manual cephalometric analysis, a created electronic cephalometric tracmgs, 
an electronic composite including an integration of the three-dimensional images, 
radiographic data (e.g., x-rays, CAT scans, MRI scans), photographs, a generated electronic 

25 articulation module, measurements and analysis of electronic models, data quality assurance 
checks, and/or supplemental data. 

The orthodontic server then generates an initial treatment from the electronic patient 
record, wherein the initial freatment plan includes precise steps to obtain a desired 
orthodontic sti^cture. In other words, the initial freatinent plan includes a series of steps, 
30 wherem, for each step, a correspondmg orthodontic apparatus will be designed to reshape 
the patieat's orthodontic stracture into a desired stinicture for the precise step. The 
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orthodontic server then transmits a digital version of the initial treatment plan to the site 
orthodontic system. Upon confcnation from the site orthodontic system, the orthodontic 
server designs an orthodontic apparatus (e.g., brackets, bands, wires, head gear, rubber band 
placement, and/or a retainer) for one of the precise steps based on the treatment plan. THe 

orthodontic aonaratua is thpn f?>h7n"r.p<-=^/oo^p~i.i_ j ^ , . , . 

_ . v-a.K,>a,^ooexiiuicd and provided to the site orthodontic 

system. In addition, the orthodontic server may provide practitioner and/or patient 
mstmctions for use, maintenance, and/or instaUation of the orthodontic apparatus. 

Upon receipt of the orthodontic apparatus, the orthodontist verifies receipt of the 
Orthodontic apparatus and subsequently instaUs it. At a predetermined point in time after 
the mstallation in accordance with the treatment plan, the patient's mouth is elecHonicaUy 
scanned to obtain updated digital infonnation. The site orthodontic system provides the 
updated digital infonnation to the orthodontic server, which uses the updated digital 
infonnation to update the electronic patient record. From the updated electronic patient 
record, the orthodontic server detennines whether the actual movement of the patient's teeth 
IS as predicted. Such a determination may be supplemented with input from a practitioner 
(e.g., a technician, dental assistant, orthodontist, specialist, consultant, etc.) atthe site or at a 
remote site with respect to the patient. If so, the next step of the.mitial treatment plan is 
executed (e.g., the whre for the next step is febricat4 provided to the orthodontist, and 
instaUed). If, however, die actual movement is not as predicted, the orthodontic server 
adjusts the freatment plan to obtain the desired results. After the treatment plan has been 
adjusted, the next step of the revised treatment plan is executed. This monitoring of a 
patient's progress and revising the treatment plan, when necessary, continues throughout the 
treatment. Thus, with such a method and apparatus, a scientific approach is provided to 
orthodontic treatment throughout the treatment, maintains the treatment costs at reasonable 
levels, and provides a more consistent and reduced freatment time. la addition, the present 
method and apparatus allow orthodontic practitioners to have just-in-time inventory, aUows 
for freatment adjustments that would be requfred due to changes in tooth position and/or jaw 
development, and finrther allows for changes in the orthodontic sfructure if the fo«:e system 
becomes sub-optimal. 
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Generally, the present invention also provides for a method and apparatus for 
producing a three-dimensional digital model of an orthodontic patient. Such a method and 
apparatus include processing that begins by obtaining data of an orthodontic stracture of the 
orthodontic patient. The processing then continues by obtaining at least one scaling 

5 reference point of the orthodontic structure. For example, flie scaling reference point may 
be a marking placed on a tooth prior to obtaining the data. The processing continues by 
scaling the data of the orthodontic structure based on the at least one scaling reference point 
to produce scaled data. Note that the data may be video data, x-rays, CAT scans, 
ultrasound, and/or magnetic resonance images. The process then continues by obtaining at 

10 least two orientation reference points relating to the orthodontic stracture (i,e., fixed 
physical attributes of the orthodontic structure , that will not change, or will change with 
negligible effects, during the course of treatment). The processing then continues by 
mapping the scaled data to a coordinate system based on at least two orientation reference 
points to produce ah enhanced ttiree-dimensional digital model of the orthodontic patient. 

15 With such a method and apparatus, the automated treatment of an orthodontic patient occurs 
in a scientific and controlled manner. By obtaining an accurate three-dimensional digital 
model that includes a dentition structure of the orthodontic patient, patient treatment can be 
smiulated by a computer in three-dimensional space, thereby providing a scientific approach 
to orthodontic treatment. 
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Producing a Desired TIiree-Dimensional Digital Model of an Orthodontic Structure 



Generally, the present invention also provides for a method and apparatus for 
generating a three-dimensional digital model of a desired orthodontic structure. Such a 
25 method and apparatus include processing that begins by obtaining a three-dimensional 
model of an actual orthodontic stracture, wherein the three-dimensional digital model is 
defined m x, y, z space. The processing then continues by generating an interim three- 
dimensional model of the desired orthodontic stracture less teeth. The interim three- 
dimensional model is designed in x, y, z space and includes the desired placement of an 
30 occlusal plane, an upper-arch form, a lower-arch form, an upper-arch midliue, and a lower- 
arch midline. The processing then continues by positioning the upper and lower teeth with 
respect to the interim digital model and the defined x, y, z space to obtain a first pass three- 
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dimensional digital model of the desired orthodontic structure. The processing continues by 
determining whether achieving the first pass three-dimensional model is feasible. In 
essence, the detemiination is based on treatment constraints and whether the desired 
orthodontic structure can be achieved from the actual orthodontic structure given such 
constraints. When achieving the first pass three-dimensional image is feasible, the 
processing continues by utihzing the first pass three-dimensional model as the desired three- 
dimensional digital model. With such a method and apparatus, automation of orthodontic 
treatment is reahzed thereby providing a scientific approach to practice of orthodontics. 

Desimin^ an Orthodontia 4DDaratii.i tn Pro\ndp. Tnpth MovRmmtt 

Generally, the present invention also provides for a method and apparatus for 
designing an orthodontic structure to provide tooth movement. Such a method and 
apparatus includes processing that begins by obtaining a three-dimensional digital model of 
a desired orthodontic stracture. The orthodontic structure includes a plurality of teeth that 
have been positioned into a desired location. The processing continues by generating a zero 
force arch wire based on the three-dimensional digital model of the desired orthodontic 
structure. The processing continues by placing brackets on at least some of the pluraUty of 
teeth in a zero force position with respect to the zero force arch wire. Having placed the 
brackets on the desired orthodontic structure, the placement is transferred to a three- 
dimensional digital model of an actual orthodontic structured Having fransferred the bracket 
placement, a jig may be fomied from the digital model of the actual orthodontic structure 
with the brackets instaUed. The jig holds the physical embodiments of the brackets such 
that the brackets may be placed in the patient's mouth in the position determined by the 
automated process. With such a method and apparatus, the practice of orthodontics is 
transfonned from an art to a science throughout the treatment process, reduces costs, and 
provides for more consistent freatment times. 

Automated Generation of a Patient TreatinRnt PJnn 



Generally,, the present invention also provides for a method and apparatus for 
;enerating a patient treatment plan. Such a method and ^aratus includes processing that 
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begins by providing a list of health care services to a patient and/or care provider. The 
processing continues by prompting for input of digital information regarding the patient 
when a health care service has been selected. The processing continues by detennining 
whether a sufficient amount of digital information has been received. If so, the processing 

5 continues by simulating treatment of a patient based on the digital information, a treatment 
objective, and normaUzed patient data. The processing then continues by generating the 
patient treatment plan in accordance with the simulating of the treatment when the 
simulated treatment results have been acknowledged. With such a method and apparatus, 
the generation of a patient treatment plan may be automated for particular types of health 

10 care services, including orthodontic care, dental care, and medical care. 

While presently preferred embodiments of the invention have been described for 
purposes of illustration of the best mode contemplated by the inventors for practicing the 
invention, wide variation from the details described herein is foreseen without departure 
15 from the spirit and scope of the invention. This tme spirit and scope is to be determined by 
reference to the appended claims. The term **bend", as used in the claims, is. interpreted to 
mean either a simple translation movement of the workpiece in one direction or a twist 
(rotation) of the workpiece, unless the context clearly indicates otherwise. 
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CLAIMS 

1 . An interactive, orthodontic care system based on scanning of teeth, comprising: 

a hand-held scanner and associated processing system for c^tuiing images of the 
dentition of the patient and processing the images to generate a fiill, > virtual, three- 
dimensional model of the dentition- 

a data conditioning system processing the virtual, three-dimensional model and 
responsiveiy generating a set of indi^adual, virtual three-dimensional tooth'' objects 
representing teeth in the dentition of the patient; 

a workstation comprising a user interface for display of said set of individual, virtual 
three-dimensional tooth objects and software adapted for pemiitting a user to manipulate the 
virtual three-dimensional tooth objects to thereby design a target situation for the patient in 
three dimensions and parameters for a customized orthodontic appUance for said teeth. 

2. The system of claim 1, wherem the hand-held scanner and associated processing 
system, said data conditioning system, and said workstation are installed in an orthodontic 
clinic. 

3. The system of claim 1, wherein the hand-held scanner and workstation are installed 
in an orthodontic cHnic, and wherein said data conditioning system is installed in a general . 
purpose computer at a remote location from said orthodontic clinic. 

4. The system of claim 3, wherein said remote location comprises an appUance 
manufacturing faciUty manufacturing said customized orthodontic appUance. 

5. The system of claim 1, further comprising communication equipment for 
transmitting digital data representing at least said target situation and said parameters for a 
customized orthodontic appliance to a remotely located appUance manufacturing faciUty, 
said appUance manufacturing faciUty receiving said digital data and further comprising 
orthodontic appUance manufacturing equipment for manufacturing said customized 
orthodontic appliance. 

6. The system of claim 1, wherein the orthodontic appliance comprises an archwire. 
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7. The system of claim 1, wherein the, orthodontic appliance comprises set of 
orthodontic brackets and an archwire, 

8. The system of claim 1, wherein the orthodontic appliance comprises a retainer. 

9. The system of claim 1, wherein the orthodontic appUance comprises a substantially 
transparent aligning deAdce. 

10. The system of claim 1, wherein the hand-held scanner is adapted for intra-oral 
10 scanning and comprises a Hght source, an electronic imaging device and a pattern source, 

said light source projecting a pattern onto said dentition, said electronic imaging device 
converting incident radiation into a two dimensional image of the pattern as reflected from 
said dentition. 

15 11. The system of claim 10, wherein the hand-held scanner further comprises a 
continuous illumination Ught source providing general illumuiation of the dentition dunng 
scanning. 

12. The system of claim 1, wherein the hand-held scanner further comprises a video 
20 mode of operation in which a Hght source illuminates the dentition and an imaging device 

generates a video image of the dentition. 

13. The system of claim 12, wherein the scanner alternates in rapid succession between 
said video mode and a three-dimensional image capture mode. 

25 

14. The system of claim 1, further comprising a scanning workstation coupled to said 
hand-held scanner wherein said scanning workstation includes a voice-activated system m 
which operation of the scanner is controlled, at least in part, by voice coxmnands. 

30 15. The system of claim 10, wherein said processing system associated with said hand- 
held scanner executes a registration dgorithm on frames of the three-dimensional point 
clouds, each frame corresponding to an individual two-dimensional image captured by said 
electronic imaging device. 
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16. The system of claim 1 5, wherein said registration algorithm comprises a cumulative 
registration algorithm. 

17. The system of claim 15, wherein said processing system associated with said hand- 
held seamier executes a pattern recognition routine for said two-dimensional images, a 
decoding routine, and a three-dimensional calculation routine generating a three- 
dimensional cloud of points associated with each of said two-dimensional images. 



18. An interactive orthodontic treatment planning system, comprising: 

a workstation having a display, a memory storing a three-dimensional virtual model 
of the deintition of a patient in a malocclused situation and a processing unit; 

said processing unit executing a routine for generating individual, virtual three- 
dimensional tooth objects of teeth from said three-dimensional virtual model of the 
1 5 dentition of a patient in a malocclused situation,- 

said processing unit further executing software representing an interactive treatment 
planning and apphance design program by which a user of said workstation may move the 
individual three-dimensional virtual tooth objects to a target situation, said target situation 
comprismg a three-dimensional vktual model of said teeth at a desired occlusion; and 

communication equipment for transmission of the three-dimensional virtual model 
of the said teeth at a desired occlusion to a remote location. 



19. The system of claim 1 8, wherein said routine for generating individual, virtual three- 
dimensional tooth models compares portions of said three-dimensional virtual model of the 

25 dentition to three-dunensional virtual template teeth. 

20. The system of claim 18, wherein said three-dimensional virtual model of the 
dentition is obtained from an intra-oral scanning of the dentition of the patient with a hand- 



held scanner. 



21. The system of claim 19, wherein the remote location comprises an orthodontic 
clinic. 



140 



wo 01/80761 PCT/USOl/11969 

22. The system of claim 1 8, wherein the remote location comprises a precision appliance 
manufacturing center. 



23. The system of claim 18, wherein the treatment planning software program furttier 
5 comprises a feature for enabUng a user to interactively place a three-dimensional virtual 
orthodontic bracket onto the surface of an individual, virtual three-dhnensional tooth model 
and adjust the position of said virtual bracket relative to said individual, virtual three- 
dimensional tooth model. 

10 24. The system of claim 23, wherein the precision apphance manufacturing center 
comprises a wire bending robot for customized shaping of an orthodontic archwire and a 
placement device for placement of orthodontic brackets at the location where the user 
placed the virtual bracket on the virtual three-dimensional tooth model. 

15 25. The system of claim 24, wherein said wire bending robot comprises a six-axis wire 
bending robot. 

26. The system of claim 24, wherein said placement device comprises a bracket 
placement tray formed over a three-dunensional physical model of the dentition of the 

20 patient in a malocclusion having orthodontic brackets placed on the surface of the teeth at a 
location specified by the user. 

27. The system of claim 26, wherein the three-dimensional physical model is made from 
a rapid prototyping method selected from the group consisting of sterehthography, laser 

25 sintering, milling, and three-dimensional printing. 

28. An interactive, orthodontic care system for use at an ortliodontic cHnic treatmg a 
patient suffering from a malocclusion, comprising: 

a hand-held, intra-oral scanner and associated processing system for capturing 
30 images of the dentition of the patient and processing the images to generate a fuU, virtual, 
three-dimensional model of the dentition in the malocclusion; 

a workstation having a display, a memory storing the three-dimensional virtual 
model of the dentition of a patient in a malocclused situation and a processing unit; 
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said workstation processing unit executing soitware representing an interactive 
treatment plamxing and appliance design program by which a user of said workstation may 
move mdividual three-dimensional, virtual tooth objects obtained from said virtual model of 
the dentition to a target situation, said target situation comprising a three-dimensional 

5 vntual model of said teeth at a desired occlusion; and 

commmiications equipment for transmission of digital infomiation representing at 
least one of said virtual, three-dimensional model of the dentition in the malocclusion and 
saad three-dimensional virtual model of said teeth at a desired occlusion to a remotely 
located precision appUance manufacturing center for manufacturing a customized 

) orthodontic appliance. 



29. 



The system of claim 28, further comprising a processing unit executing a routine for 
generatmg individual, virtual three-dimensional tooth models of teeth from said three- 
chmensional virtual model of the dentition of a patient in a malocclused situation,- 

30. The system of claim 29, wherein said processing unit comprises the processing unit 
of said workstation. 



31. The system of claim 29, wherein said processing mnt comprises a general-puipose 
computer at said precision appUance manufacturing center. 

32. The system of claim 29. wherein the orthodontic appliance comprises an archwire. 

33. The system of claim 28, wherein the orthodontic appUance comprises one or more 
devices for placement of orthodontic brackets and an archwhre. 



34. The system of claim 28. wherein the orthodontic appUance 



compnses a retainer. 



35. The system of claim 28. wherein the orthodontic appUance comprises a removeable 
aligning device. 

36. The system of claim 28, wherein the hand-held seamier includes a Ught source, an 
elecfronic imaging device and a pattern source, said Ught source projecting a pattern onto 
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said dentition, said electronic imaging device converting incident 'v^ciiation into a two 
dimensional image of the pattern as reflected from said dentition. 

37. The system of claim 36, wherein the hand-held scanner further comprises a 
continuous illumination light source providing general illumiiiation of the dentition during 
scanning. 



38. The system of claim 28, wherein the scanner further comprises a video mode of 
operation in which a Hght source iUuminates die dentition and an imaging device generates 

10 a video image of the dentition. 

39. The system of claim 38, wherein the scanner alternates in rapid succession between 
said video mode and a three-dimensional image capture mode. 

15 40. The system of claim 38, further comprising a scanning workstation coupled to said 
hand-held scanner wherein said scanning workstation includes a voice-activated system m 
which operation of the scanner is controlled, at least m part, by voice commands. 

41. The system of claim 36, wherein said processing system associated with said hand- 
20 held scanner executes a registiration algorithm on frames of tiie three-dimensional pomt 

clouds, each frame corresponding to an mdividual two-dimensional image captured by said 
electronic imaging device. 

42. The system of claim 41 , wherein said registration algoritiim comprises a cumulative 
25 registration algorithm. 

43. The system of claim 42, wherein said processing system associated with said hand- 
held scanner executes a pattern recognition routine for said two-dimensional images, a 
decoding routine, and a tiiree-dimensional calculation routine generating a tiaree- 

30 dimensional cloud of points associated witii each of said two-dimensional images. 



44. An orthodontic appliance manufacturing system, comprising: 
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a machine readable memory storing digital data representing a three-dimensional 
virtual model of a malocclusion of a patient and digital information representing the 
location of orthodontic brackets to be placed on the malocclusion, said memory further 
storing digital data representing a three-dimensional virtual model of the patient's teeth and 
the dentition and location of orthodontic brackets at a target situation in three dimensions; 

a wire bending robot; 

an instrument for generating a three-dimensional physical model of the malocclusion 
with the brackets placed on the malocclusion; 

a fonning device fonning a bracket placement tray matching the geometry of the 
physical model of the malocclusion with the brackets; 

wherein the bracket placement tray may be removed from the physical model, said 
tray fonned with spaces for placement of brackets and enabhng bonding of the brackets to 
the teeth at a desired location based on the virtual model of the dentition and the placement 
of orthodontic brackets on the malocclusion. 



45. The system of claim 45, wherein the wire bendmg robot comprises a six axis robot 
having a fixed gripping tool and a moveable gripping tool mounted to the end of a moveable 
arm. 

46. The system of claim 45, wherein the instrument for generating the three-dimensional 
model comprises a stereolithographic instrument. 

47. The system of claim 45, further comprising equipment for manufacturing a 
customized bracket bonding pad based on the placement of a three-dimensional virtual 
orthodontic bracket on the surface of a three-dimensional virtual model of the dentition of a 
patient. 

48. A system for manufacture of a customized bracket bonding pad for a bracket, the 
bracket having a base, comprising: 
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a memory storing a three-dimensional virtual model of the dentition and a three- 
dimensional virmal model of an orthodontic bracket- as placed on the surface of a tooth in 
the three-dimensional virtual model of the dentition; 

a processing unit deriving from said three-dimensional virtual models the three- 
5 dimensional shape of a bracket bonding pad; and 

a machine system for manufacturing a bracket bonding pad according to said denved 
three-dimensional shape. , . 

49. A machine for bending a medical device into a desired configuration, comprising 
10 a first gripping tool and a second gripping tool, each of said gripping tools having a 

gripping structure for releasably holding said medical device; 

a moveable arm having a free distal end portion adapted to mount said second 
gripping tool to said moveable arm; 

said two moveable arm capable of independent movement relative to the first 
15 gripping tool about at least two different axes; 

a control system operative of said of the moveable arm and the gripping tools to grip 
the medical device while said gripping tools are separated from each other and to cause the 
gripping tools to move relative to each other about at least two axes to thereby bend said 
medical device a desired amoimt. 

20 

50. The apparatus of claim 49, wherein said moveable arm is capable moving relative to 
said first gripping tool about three translation axes and three rotation axes. 

25 51. The apparatus of claim 49, wherein said apparatus further comprises a force sensor 
system for detecting forces generated by said medical device after said medical device has 
been bent by said first and second gripping tools. 

52. The apparatus of claim 49, wherein said force system comprises a first force sensor 
30 coupled to said first gripping tool and a second force sensor coupled to said second 

gripping tool. 



53. The apparatus of claim 49, wherein said medical device comprises an orthodontic 
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appliance. 



54. The apparatus of claim 49, wherein said medical device comprises a fixation plate. 

55. The.apparatus of claim 49, wherein said apparatus further comprises a heating system 
for applying heat to said medical device while said medical device is being bent by said first 
and second gripping tools. 



56. A scanner system for capturing three-dimensional information of an object, 
comprising: 

a projection system projecting a pattern onto said object along a first axis; 

an electronic imaging device comprising an array of pixels, said electronic imaging 
device oriented along a second axis different from said first axis, said electronic imaging 
device forming an image of said pattern after reflection of said pattern off of said object; 

a memory storing a three-axis (X, Y and Z) coordinate system calibration 
relationship for said scanner, said calibration relationship identifying: (1) pixel coordinates 
for said electronic imaging device for numerous portions of said pattern, said pixel 
coordinates associated with distance information from said projection system in a Z 
direction at at least two different Z distances, and (2) distance information in X and Y 
directions, for said numerous portions of said pattern, at said at least two different Z 
distances; and 

at least one processing unit processing said image of said pattern and comparing data 
from said image to said calibration relationship to thereby derive spatial mformation in three 
dimensions of points on said object reflecting said projected pattern onto said electironic 
imaging device. 



57. The scanner system of claim 56, wherein said processing system derives said spatial 
information in three dimensions from said calibration relationship without use of optical 
characteristics of lens systems associated with said projection system and said electironic 
imaging device, to thereby compensate for optical distortions present in said lens systemi. 

58. A method of obtaming three-dimensional surface information of an object, 
comprising the steps of: 
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positioning a scanner proximate to said object, said scanner having a projection 
system for projecting a pattern onto said object and an electronic imaging device generating 
two dimensional images of the reflection of said pattern off of said surface; 
moving said scanner and object relative to each other; 

capturing, with said electronic imaging device, a series of two dimensional images 
of said projection pattern after reflection of said pattern off of said object as said scanner 
and said surface move relative to each other, 

processing said series of images to obtain a series of frames of said object, each 
frame comprising spatial coordinates for a portion of the surface of said object in three 
dimensions, and 

registration overlapping areas of said frames to each other to generate a three- 
dimensional virtual model of said object. 

59. The method of claim 58, wherein said series of images are captured at a rate of at least 
one unage per second and wherein said pattern is projected onto said object in a senes of 
flashes timed at a rate in synchronism with said image capture rate. 

60. The method of claim 57, . wherein after initiation of said series of flashes said flashes 
continue automatically without human intervention. 

61 . The method of claim 57, wherein said step of processing further comprises a step of 
executing a decoding algorithm identifying specific portions of said projected pattern 
captured by said electronic imaging device after reflection from said gbj ect. ■ 

5 62. The method of claim 57, further comprising the step of storing a caUbration 
relationship for said scanner wherein said processing system derives said spatial iriformation, 
in three dimensions from said caUbration relationship without use of optical characteristics 
of lens systems associated with said projection system and said electronic imaging device, 
to thereby compensate for optical distortions present in said lems,system. . 



[) 
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63. The method of claim 62, wherein said calibration relationship comprises a 
mathematical function providing Z distance information and X and Y. values .from mput 
comprising pixel addresses in said electronic imaging device. 
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64. The method of claim 63, wherein said calibration relationship comprises a 
calibration table. 

65. The method of claim 57, further comprising the step of providing a data storage 
medium storing two dimensional images obtained by said electronic imaging device needed 
to cover said object. 

65. The method of claim 57, wherein said step of processing comprising the steps of: 

a) performing pattern recognition of said series of images to detect features 
of said projected pattern in each of said images, 

b) decoding of patterns detected by said pattern recognition step to identify 
specific portions of said projected pattern c^tured by said electronic 
imaging device; and 

c) calculation of spatial coordinates in three dimensions for said specific 
portions of said decoded patterns. 

66. The method of claim 65, wherein said step of calculation comprises comparing 
decoded points of said projected pattern captured by said electronic imaging device with 
caHbration values stored in a calibration table stored in a memory for said scanner. 

67. The method of claim 65, wherein each two-dimensional image captured by said 
electronic imaging device is converted to a frame in accordance with said calculation step, 
and wherein said series of frames are stored in a memory, and wherein the method further 
comprises the step of executing a registration algorithm to register said series of firames to 
derive a three-dimensional model of the surface of said object. 

68. A method of constructing a virtual three-dimensional model of an object firom a 
scanner, a data processing system, and at least one machine-readable memory 
accessible to said data processing system, comprising the steps of: 

(a) scanning the object with the scanner and thereby obtaining at least two two- 
dimensional images of the object, wherein during scanning the scanner and object are 
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moved relative to each other resulting in each image being taken from a different position 
relative to the surface of the object; 

(b) processing said data representing said set of images with said data processing 
system so as to convert each of said two-dimensional images into a data representmg a 

5 frame and thereby generate a set of frames corresponding to said images, said set of frames 
comprising a cloud of individual points, each point in each frame expressed as a location in 
a three-dimensional coordinate system; 

(c) storing data representing said set of frames in said memory; and 

(d) farther processing said data representing said set of frames with said data 
10 processing system so as to register said frames relative to each other to thereby produce a 

three-dimensional virtual model of the object substantially consistent with all of said 
frames. 

69. The method of claim 68, wherein step (d) comprises the step of performing a 
15 frame to frame registration of said set of frames, wherein each frame is registered with 

respect to one other frame in said set of frames. 

70. The method of claim 68, wherein step (d) comprises the step of performing a 
cumulative registration of said set of frames, wherein at least some of said frames are 

20 registered to a plurality of other frames previously having been registered to other frames m 
said set of frames. 

71. The method of claim 68, wherein step (d) comprises performing a frame to 
frame registration of said set of frames, wherein each frame is registered with respect to one 

25 other frame in said set of frames, and thereafter performing a cumulative registration of said 
frames wherein at least some of said frames are registered to a pluraUty of other frames 
previously having been registered to other frames ia said set of frames. 

72. The method of claim 68, wherein in step (d) one of said frames is a starting 
30 frame for regisfration, and wherein a spatial transformation relationship is derived for each 

of the other frames in said set of frames and stored in said memory, said spatial 
fransformation relationship indicating how the pomts in said frame should be translated and 
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rotated in a three-dimensional coordinate system to register said frames relative to said 
starting frame. 



73. The method of claim 72, wherein said starting frame coiresponds to the first 
image captured by said scanner. 



74. The method of claim 72, wherein said starting frame corresponds to selected 
image taken of the object and in which other images were taken of the object in the same 
vicinity of the object such that a substantial amount of overlap exists between said selected 
image and said other images. 



75. The method of claim 68, wherein, in step (d), .said set of frames are 
registered to each other in a sequential order with the order determined, at least in part, upon 
the degree of overlap in coverage of said object in said frames. 

76. The method of claim 68, wherein said scanner comprises a hand-held 
scanning device and said object is scamied by moving said hand-held scanning device over 
said object. 



77. The method of claim 68, wherein said object comprises ahuman. 

78. the method of clahn 68, wherein said object comprises teeth and associated 
anatomical structures. 



79. The method of claim 78, wherein said object comprises a model of an 
anatomical structure of a human. 

80. The method of claim 68, wherein said data processing system is incorporated 
into a work station for said scanner. 

81. Treatment planning apparatus for an orthodontic patient, comprising: 
a workstation having a processmg unit and a display; 
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a memory storing a virtual tbree-dimensional model of teeth and/or associated 
anatomical structures representing the dentition of a patient; aad 

software executable by said processing unit to access said model and display said 
model on said display; . 

said software further including navigation tools enabling a user to interactively 
manipulate said model on said display and simulate the movement of at least one tooth in 
said model relative to other teeth in said model. 

82. The apparatus of claim 81, further comprising: 
a library of virtual, three-dimensional orthodontic brackets, said software pemiitting 
a user to access said virtual brackets, place said ^drtual brackets on said virtual model of the 
dentition of the patient, and simultaneously display both said virtual brackets and said 
virtual model. 

83. The apparatus of claim 81, wherein said virtual model of teeth comprises a 
set of virtual, individual three-dimensional tooth objects and wherein individual ones of said 
virtual tooth objects can be individually selected and moved relative to otlier teeth in said 
set of virtual tooth obj ects. 

84. The apparatus of claim 81, wherein said virtual model comprises a set of 
virtual, mdividual three-dimensional tooth objects and wherein said software enables a user 
of said workstation to select an archform for said virtual model, wherein said set of tooth 
objects are moved with respect to said archform. 

85- The apparatus of claim 81, wherein said virtual model comprises a set of 
virtual, individual tbree-dimensional tooth objects and wherein said set of tooth objects are 
moved with respect to each other to define an archform representing, at least in part, a 
proposed treatment for the patient. 

86. The apparatus of claim 84 or claim 85, wherein said software displays a 
screen pemiitting a user to modify the position of said individual tooth objects with respect 
to said at least one of the following: said archform, associated anatomical structures, and 
other teeth. 
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87. A system for creating an individual, three-dimensional virtual tooth model 
representing a tooth found in a dentition assigned to a patient, comprising: 

a memory storing at least one virtual three-dimensional model of a template object 
corresponding to a tooth, including a template tooth corresponding to said tooth; 

a data processing system including a memoiy storing a virtual three-dimensional 
model of at least a part of the dentition; 

wherein said data processing system further comprises software processing said 
virtual three-dimensional model of at least a part of the dentition and said virtual model of 
the template object and responsively deriving said individual, three-dimensional, virtual 
tooth model. 

88. The system of claim 87, wherein said template object comprises one or more 
specific characteristics of a tooth selected &om the group consisting of cusps, tooth width, 
occlusion characteristics, two-dimensional cross-section in an occlusal plane, roots and 
flatness. 

89. The system of claim 87, wherein said template object comprises a surface 
formed of a mesh of polygpn surfaces. 

90. The system of claim 87, wherein said template object comprises a cloud of 
. three-dimensional points; 

91. , .An orthodontic workstation, comprising: 

• a memory comprising data storage regions storing a data representing virtual three- 
dimensional anatomical structures including an individual, three-dimensional virtual tooth 
model representing an individual tooth of a patient isolated from surrounding anatomical 
structures; 

a u^er interface including a display for displaying said individual virtual tooth 

model; 

interactive software for aUowing a user to manipulate said virtual tooth model with 
respect to said surrounding anatomical stractures. 
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92. The workstation of claim 91, wherein said individual virtual tooth model is 
generated from a scan of the dentition of a patient and a virtual three-dimensional template 
tooth. 

93. The workstation of claim 9L whereia said individual, three-dimensional 
virtual tooth model further comprises a root for said tooth, said root derived from a virtual, 
three-dimensional template tooth root. 

94. The workstation of claim 91, wherein said memory further comprises data 
storage regions storing a pluraUty of virtual, three-dimensional template orthodontic 
brackets. 

95. The workstation of claim 94, wherein said workstation further comprises 
software for permitting a user to place one of said virtual, three-dimensional template 
orthodontic brackets onto the surface of said individual, virtual three-dimensional tooth 
model and display said placed template bracket on said mdividual, virtual three-dmaensional 
tooth model. 
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